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PREFACE 


The  simulation  experiment  described  herein  is  the  third  of  a  multiple  experiment 
program  for  the  operational  evaluation  of  radio  aids  to  navigation  displays.  The 
intent  of  the  overall  program  is  to  investigate  navigational  safety  as  a  function  of 
display  cost,  complexity  and  system  error  characteristics.  The  results  will  define 
the  requirements  for  an  electronic  display  which  will  allow  safe  pilotage  of  vessels 
in  poor  visibility  conditions  keeping  in  mind  all  economic,  technological  and 
feasibility  constraints. 

The  report  briefly  reviews  the  conduct  and  results  of  each  of  the  previous 
experiments  which  led  to  the  present  RA-2  experiment.  It  also  describes  in  detail 
the  RA-2  simulator  evaluation  which  consisted  of  sixty-four  35-minute  runs  to 
determine  die  effects  of  electronic  navigation  system  noise,  tracking  filter  design 
and  tracking  filter  aiding  technique  on  restricted  waterway  pilotage.  Using  the 
"benchmark”  true  motion,  track-up  GRAPHIC  display  recommended  in  the  RA-1 
experiment,  the  RA-2  evaluation  was  conducted  in  controlled  levels  of  random  noise 
to  simulate  potential  real  world  operating  conditions.  An  ALPHA-BETA  tracker 
with  three  preselected  rise  times  and  gyro  aiding  was  introduced  to  represent  the 
potential  capability  of  a  state-of-the-art  navigation  system.  With  the  exception  of  a 
shortened  first  leg  of  the  waterway,  conduct  of  the  RA-2  experiment  was  identical 
to  the  RA-1  experiment.  Some  performance  measures  which  were  shown  in  the  RA- 
1  experiment  to  be  ineffectual  for  examining  pilotage  behavior,  are  omitted  in  the 
RA-2  analysis.  Results  of  the  entire  program  including  recommendations  for  display 
design,  system  implementation,  and  additional  experimentation  are  presented. 

"The  ultimate  objective  of  the  program  will  be  realized  by  a  combination  of  the 
performance  metric,  the  various  signal-to-noise  ratios,  and  filter  bandwidths  into  a 
definitive  statement  about  the  ability  of  a  pilot  to  navigate  a  restricted  waterway  in 
limited  visibility  conditions." 


*  United  States  Coast  Guard.  An  Approach  to  the  Study  of  Electronic  Displays  for 
Use  in  Restricted  Waterways,  a  Position  Paper.  December  1979. 
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SECTION  1 

SUMMARY  OF  CONCLUSIONS 

This  summary  of  conclusions  is  derived  from  a  review  of  the  entire  simulator 
evaluation  of  electronic  radio  aids  to  navigation  displays  project.  The  review 
included  the  miniexperiment  and  RA-1  experiments  which  presented  ownship  on  the 
display  using  "perfect"  position  information.  These  experiments  were  conducted  to 
select  a  "benchmark"  display  for  further  evaluation.  The  review  also  included  the 
RA-2  experiment  reported  herein  which  was  conducted  to  determine  the  effects  of 
noise  environment  and  tracker  filter  design  on  the  benchmark  display.  Evidence  in 
support  of  these  conclusions  as  well  as  a  detailed  description  of  the  experimental 
variables  can  be  found  in  the  respective  interim  reports.  Evidence  in  support  of  RA- 
2  conclusions  is  presented  in  Section  5  of  this  report. 

1.1  EFFECT  OF  DISPLAY  FORMAT 

Selection  of  a  format  design  for  the  benchmark  display  was  based  upon  (1)  its 
safety  and  operational  effectiveness  for  the  conduct  of  restricted  waterway  pilotage 
under  poor  visibility  conditions,  (2)  its  acceptance  as  an  aid  to  navigation  by 
potential  users,  and  (3)  its  cost-effective  implementation  and  usage.  Results  of  the 
initial  experiments  showed  that  while  certain  formats  excelled  in  one  or  the  other 
criteria,  only  the  GRAPHIC  display  was  deemed  favorable  in  all  of  them.  Table  1 
presents  the  categories  of  displays  which  were  evaluated  along  with  a  brief 
description  of  the  deficiencies  and  attributes  which  were  revealed.  Displays  are 
listed  in  their  order  of  preference  with  the  first  (GRAPHIC)  being  recommended  as 
the  benchmark  display. 

As  a  result  of  the  pragmatic  approach  adopted  by  this  project  and  an 
appreciation  that  the  final  RA-2  experiment  should  employ  the  display  with  the 
"best  chance  for  success",  the  true  motion,  track-up  GRAPHIC  display  with  a 
heading  vector  and  own  ship's  scaled  image  was  selected. 

1.2  EFFECT  OF  NOISE  AND  TRACKER  DESIGN 

Simulation  of  the  benchmark  display  as  though  it  were  being  used  with  an  actual 
navigation  system  produced  a  previously  unaddressed  problem;  the  effects  on 
pilotage  of  display  lag  and  jitter.  A  detailed  description  of  the  causes  for  display  lag 
and  jitter  and  their  relationship  to  system  noise  levels  and  jitter  design  is  presented 
in  Section  2.3. 

Briefly,  jitter  is  the  "jumping  around"  of  own  ship  on  the  display  as  a  function  of 
random  position  errors  from  the  navigation  system.  As  noise  to  the  navigation 
system  increases,  so  does  the  magnitude  of  the  jitter.  Mathematical  tracker  filters 
car.  be  introduced  into  the  system  to  reduce  the  jitter  effects  of  noise.  They, 
however,  produce  a  display  lag  which,  once  ownship  begins  to  maneuver,  produces  a 
bias  error  proportional  to  their  filtering  capability.  The  better  the  tracker  is  at 
noise  filtering,  the  larger  its  potential  lag.  To  compensate  for  the  lag  deficiency, 
tracker  filters  can  be  aided  by  gyro  compass  inputs  which  modify  their  filter 
computation.  This  in  turn  enables  the  tracker  to  produce  minimal  display  jitter  from 
noise  and  minimal  lag  during  a  maneuver;  both,  however,  at  the  potentially 
substantial  cost  of  a  gyro  input. 
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TABLE  1.  CONCLUSIONS  Ul  Tilt  DISPLAY  HOKMAT  EVOLUTION  (CONT.) 


Very  low  user  acceptance  primarily  the  result  of  insufficient  experience 
with  the  display;  but  also  the  premise  that  no  pilot  would  attempt  this 
maneuver  in  the  real  world  with  such  scant  information. 


Table  2  summarizes  pilotage  performance  as  a  function  of  the  trade-offs  in  noise 
and  tracker  design.  It  suggests  that  use  of  the  electronic  radio  aids  to  navigation 
display  alone  never  achieved  trackkeeping  or  maneuvering  performance  comparable 
to  those  simulations  in  which  pilots  could  view  out  the  windows.  There  is  evidence, 
however,  that  if  pilots  were  given  the  opportunity  to  "prove"  the  display  during  good 
visibility,  and  if  the  display  design  was  made  somewhat  more  effective,  the  system's 
safe  use  under  these  operating  conditions  could  be  assured. 

The  following  conclusions  are  derived  from  Table  2  and  the  detailed  analysis 
described  in  Section  5: 

•  Those  same  difficulties  in  negotiating  the  bend  during  the  previous  experi¬ 
ments,  namely,  a  tendency  to  overshoot  the  bend  and  high  crosstrack  variability  in 
steadying  up,  were  also  in  evidence  during  the  best  pilotages  with  simulated  noise 
and  tracker  characteristics. 

•  The  lag  characteristic  of  trackers  in  this  scenario  made  it  appear  that 
pilotage  performance  was  actually  improved.  This  lag  caused  pilots  to  apply  more 
rudder  than  they  normally  would,  thus  reducing  the  characteristic  overshoot  problem 
discussed  above.  Although  it  appeared  from  the  tracks-made-good  that  the  pilotage 
was  superior;  the  fact  that  pilots  were  not  where  they  thought  they  were,  and  that 
they  had  experienced  considerable  anxiety  over  the  ship's  lack  of  initial  response 
indicates  a  relatively  unsafe  pilotage.  Further,  were  the  display  made  more 
effective,  or  were  a  different  scenario  used  (e.g.,  different  speed  or  type  of 
maneuver)  this  particular  effect  of  lag  might  not  occur. 

•  Lag  resulting  from  tracker  design  was  shown  to  have  more  deleterious  effects 
on  overall  pilotage  performance  than  jitter  from  noise  for  the  variables  and 
conditions  tested  in  the  experiment. 

•  The  relatively  large  lag  exhibited  by  the  trackers  with  24-second  rise  time 
produced  difficulty  in  all  aspects  of  maneuvering.  This  was  evidenced  in  the  pilots' 
perceived  shiphandling  difficulties  as  well  as  a  potential  for  major  undershoot  of  the 
bend. 

•  The  moderate  lag  exhibited  by  trackers  with  12-  second  rise  times  produced 
potential  for  undershooting  the  bend,  some  difficulty  recovering  from  the  turn 
maneuver,  but  more  importantly,  there  was  less  pilot  anxiety  through  the  maneuvers 
and  relatively  good  trackkeeping  as  a  result  of  minimal  jitter. 

•  For  pilots  who  negotiated  the  bend  keeping  ownship  in  the  center  of  the 
channel  on  the  display  (i.e.,  their  best  performance),  the  effects  of  lag  caused  their 
ship  to  pass  inside  the  bend,  steadyirtg  up  on  the  left  side  of  the  channel,  and  very 
gradually  returning  to  the  centerline.  In  the  event  of  approaching  traffic,  this 
maneuver  is  potentially  unsafe. 

•  The  effects  of  tracker  lag  are  virtually  unknown  to  the  pilot  except  for  the 
perceived  unpredictable  handling  characteristics  of  the  ship  during  a  maneuver.  This 
results  in  the  track- made-good  being  off-set  from  the  intended  track  for  a  distance 
approximately  equivalent  to  the  lag  error  and  in  the  direction  opposite  the  lag  error. 
If  the  lag  error  were  biased  to  the  outside  of  the  turn  maneuver  (such  as  occurred  in 
this  experiment),  ownship  would  actually  track  inside  (i.e.,  undershoot)  the  bend, 
unbeknownst  to  the  pilot. 


TABLE  2.  SUMMARY  OF  PILOTAGE  PERFORMANCE  BY  TRACKER  DESIGN 


•  When  lag  occurs  during  a  maneuver  such  as  that  simulated,  ownship  is 
perceived  by  the  pilot  as  (1)  responding  sluggishly  to  initial  turn  rudder  and  any 
subsequent  larger  turn  rudder,  (2)  responding  excessively  to  check  rudder  and 
subsequent  steady-up  rudder,  and  (4)  responding  sluggishly  again  in  the  event  a  large 
steady-up  rudder  is  required. 

•  Jitter  of  ownship  on  the  display  was  visibly  obvious  proportional  to  the 
amount  of  noise  and  filtering  capability  of  the  tracker.  Since  pilots  were  unaware  of 
lag  effects,  they  related  jitter  to  positioning  accuracy  and  assumed  this  "accuracy" 
to  be  the  major  difference  between  displays.  If  users  are  required  to  be  aware  of  lag 
and  its  effects,  the  users  must  be  given  an  opportunity  to  "prove"  the  display.  This 
can  be  accomplished  through  exercises  in  which  ownship's  position  on  the  display  can 
be  continuously  compared  with  its  actual  position  in  the  waterways. 

•  Ownship  jitter  as  a  result  of  noise  was  the  sole  contributor  to  trackkeeping 
difficulties  which  occurred;  although  in  ail  the  noise  conditions  and  for  all  the  filter 
designs  tested,  trackkeeping  performance  was  considered  to  be  adequate  and  safe. 

•  Tracker  lag  as  a  result  of  tracker  rise  time  was  the  major  contributor  to 
maneuvering  difficulties  which  occurred.  In  cases  of  long  tracker  rise  time  in  high 
noise  conditions,  the  benefits  of  reduced  jitter  did  not  overcome  the  detriments 
introducted  by  lag. 

•  The  combined  effects  of  noise  and  tracker  design  were  manifest  in  (1) 
subjects'  perception  of  system  accuracy  resulting  from  ownship  jitter,  (2)  subjects' 
perception  of  shiphandling  difficulties  when  maneuvering  and  (3)  a  major  crosstrack 
bias  between  where  subjects  thought  they  were  and  where  they  actually  were. 

•  While  pilots  were  able  to  "visually  filter"  even  the  largest  jitter  (i.e.,  a 
tracker  with  3-second  rise  time  in  32  meter  rms  noise)  without  significant 
degradation  in  trackkeeping;  their  apprehension  to  do  this  is  reflected  by  more 
conservative  maneuvers  and  very  negative  responses  pertaining  to  their  own 
performance  and  the  display's  usefulness. 

•  Subjects'  perception  of  the  jitter  was  linear  from  minimal  on  the  longest  rise 
time  trackers  in  low  noise,  to  extreme  on  the  shortest  rise  time  tracker  in  high 
noise.  There  was  no  clear  demarcation  as  to  when  it  became  an  annoyance  or 
difficulty  for  them. 

•  Learning  to  use  the  display  in  the  noise  and  operational  environment  had  no 
significant  effect  on  overall  pilotage.  This  is  attributed  to  a  lack  of  feedback  on 
ownship's  actual  status  and  its  position  in  the  waterway. 

•  Subjects'  overall  pilotage  performance  and  user  acceptance  indicates  the 
following. 


1.  Due  primarily  to  jitter,  the  tracker  with  3-second  rise  time  is  unaccept¬ 
able  from  the  users'  standpoint  in  the  high  noise  level,  but  acceptable  from  a 
performance  standpoint  in  both  noise  levels. 

2.  Due  primarily  to  lag,  the  tracker  with  24-second  rise  time  is  unacceptable 
both  from  a  user's  standpoint  and  from  a  performance  standpoint. 
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3.  The  tracker  with  12-second  rise  time  is  acceptable  from  a  user's 
standpoint  and  a  performance  standpoint  in  both  noise  levels.  The  potential  for  bias 
error  exists  as  a  result  of  tracker  lag. 

4.  The  gyro-aided  tracker  with  24-second  rise  time  is  acceptable  from  all 
aspects  and  has  minimal  potential  for  bias  error. 


Section  2 
INTRODUCTION 


As  a  result  of  technological  advances  in  electronic  information  processing, 
integration  and  display,  new  design  alternatives  for  shipboard  navigation  systems 
that  were  previously  considered  too  expensive,  inaccurate  or  unreliable  for  the 
maritime  industry  are  emerging.  The  U.S.  Coast  Guard  in  its  endeavor  to  ensure 
safe  pilotage  of  vessels  in  restricted  waterways,  yet  accommodate  cost-effective 
ship  operations,  is  undertaking  several  programs  specifically  to  develop  and  evaluate 
such  alternatives.  The  Coast  Guard  recognizes  that  to  effectively  develop  and 
implement  an  electronic  navigation  system  for  use  in  restricted  waterways,  both 
technological  advantages  and  wide  acceptance  by  potential  users  must  be  achieved. 
To  accomplish  this,  performance  and  design  requirements  in  support  of  operational 
effectiveness,  safety,  and  cost  are  required.  This,  in  turn,  has  led  the  Coast  Guard 
to  engage  in  two  parallel  but  diverse  research  efforts. 

The  first  effort,  under  contract  to  the  Applied  Physics  Laboratory  (APL)  of  Johns 
Hopkins  University,  is  for  development,  fabrication,  installation,  and  shipboard 
evaluation  of  the  precision  intracoastal  loran  translocation  (PILOT)  navigation 

system.^  This  system  specifically  utilizes  the  mini-LORAN  C  chain  located  on 
the  5t.  Mary's  River,  the  Great  Lakes  region.  While  the  PILOT  experiment  makes 
use  of  state-of-the-art  processing,  data  storage  and  display  technology,  its  primary 
function  is  to  evaluate  information  processing  and  user  acceptance  within  the 
LORAN  C  system.  Several  units  have  been  installed  on  ships  which  frequent  the  St. 
Mary's  River.  Bridge  personnel  are  requested  to  use  the  equipment  at  their 
convenience  and  report  their  results.  Experimental  limitations  result  from  the 
system's  design  .and  the  daily  environment.  The  PILOT  display  format  which  was 
engineered  based  upon  previous  research^  is  not  easily  modified  or  reconfigured. 
Difficulties  in  achieving  experimental  control  with  the  PILOT  system  as  it  is 
deployed  precludes  using  it  to  investigate  the  effects  of  design  alternatives  on  pilot 
performance.  It  does,  however,  appear  to  be  receiving  constructive  criticism 
through  exposure  and  continued  use,  the  purpose  for  which  it  was  intended. 

As  a  result  of  modern  miniaturization,  high  speed,  and  mass  storage  computer 
capabilities,  it  is  possible  for  a  shipboard  system  to  provide  accurate  positioning 
information  from  a  variety  of  navigation  sources.  This  information  can  be  processed 
using  different  filtering  or  tracking  techniques  and  can  be  presented  to  the  pilot  in 
the  most  usable  format.  All  of  these  variables,  of  course,  affect  overall  operational 
effectiveness.  It  is  this  operational  effectiveness  which  must  be  defined  and 
understood  so  that  intelligent  tradeoffs  in  the  design  of  the  pilot’s  navigation  system 
can  be  achieved.  Since  the  PILOT  experiment  cannot  vary  system  and  display 
parameters,  a  parallel  program  for  methodically  examining  their  effects  using  highly 
controlled  simulation  was  conducted.  This  simulation  evaluation  is  the  subject  of 
this  report. 


C.  R.  Edwards  and  3.  M.  Ligon.  "PILOT:  Precision  Intracoastal  LORAN 
Translocation  -  Exploiting  LORAN-C  in  the  Harbor  and  River  Environment."  Paper 
delivered  at  the  Wild  Goose  Association  Eighth  Annual  Technical  Symposium, 
Williamsburg,  Virginia,  October  1979. 


2Ibid. 
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The  program  known  as  the  "Simulator  Evaluation  of  Electronic  Radio  Aids  to 
Navigation  Displays,"  was  conducted  in  three  individual  experiments,  all  employing 
the  same  simulator  and  simulation  scenario. 

The  program  implicitly  addressed  the  restricted  waterway  environment  in  which 
the  pilot  is  faced  with  a  plethora  of  task  demands.  Here,  reduced  visibility,  the 
removal  or  relocation  of  floating  aids,  as  well  as  pressures  of  communication, 
collision  avoidance,  channel  maneuvering  and  traffic  regulations  all  continuously 
encumber  him.  In  this  environment,  the  electronic  navigation  system  must  supplant 
the  visual,  providing  immediate  information  not  only  about  present  position,  but 
about  future  position,  maneuver  timing  and  vessel  motion.  The  navigation  system, 
electronic  processing,  display,  ship,  and  environment  all  function  together  in  a 
complex  interaction  which  at  present  cannot  be  modeled.  This  research  used 
simulation  as  a  means  of  parametrically  and  functionally  studying  certain  features 
of  the  shipboard  system  that  are  design  controllable  or  specifiable. 

The  ultimate  objective  of  the  program  is  realized  by  a  combination  of  the 
performance  metric,  various  signal  to  noise  ratios  and  filter  bandwidths  into  a 
definitive  statement  about  the  ability  of  a  pilot  to  navigate  a  restricted  waterway  in 

limited  visibility  Conditions."* 

Figure  1  shows  the  sequence  of  tasks  required  to  fulfill  the  goals  of  the  program. 
Specifically,  three  experiments  were  conducted  each  dependent  upon  the  results  of 
the  previous  one.  The  miniexperiment  compared  pilotage  performance  for  18 
different  display  formats  of  navigation  information  using  an  abbreviated  simulation 
scenario.  These  displays  were  designed  in  accordance  with  traditional  human 
engineering  criteria.  From  the  miniexperiment,  five  of  the  most  effective  displays 
were  selected  and  two  more  added  to  a  full  length  RA-1  experiment.  Both  the 
miniexperiment  and  RA-1  experiment  were  conducted  in  a  "no  noise"  environment. 
Ownship  was  shown  on  the  display  in  the  exact  same  position  as  it  was  in  the  real 
world.  This  is  known  as  "perfect"  position  information.  The  single  most  effective 
display  from  the  RA-1  experiment  was  selected  as  the  benchmark  display.  This 
display  was  further  evaluated  in  the  RA-2  experiment  which  simulated  real  world 
noise  conditions  and  different  tracker  filtering  techniques.  Effects  on  pilotage 
performance  which  were  seen  in  the  RA-2  experiment  are  attributed  to  jitter  ar.d 
lag  of  ownship  on  the  display.  These,  of  course,  were  the  result  of  the  different 
noise  levels  and  tracker  characteristics. 


Coast  Guard.  "An  Aooroach  to  the  Study  of  Electronic  Displays  for 
'-se  :n  Restricted  Waterways."  A  position  paper.  December  1979. 
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Figure  1.  Program  for  the  Simulation  Evaluation  of 
Electronic  Radio  Aids  to  Navigation  Displays 


2.1  THE  MINIEXPERIMENT 


*N 


As  a  result  of  preliminary  critiques  ot  the  PILOT  system,  it  was  recognized  that 
information  provided  by  a  navigation  system  for  use  by  pilots  is  only  as  effective  as 
the  human  interface  through  which  the  information  is  presented.  For  a  display  to  be 
beneficial  during  pilotage  and  thereby  accepted  by  pilots,  it  must  be  easily 
understood,  relevant  to  the  immediate  task,  appear  clear  and  concise  within 
perceptual  limits,  and  instill  confidence  in  the  user.  It  was  further  recognized  that 
independent  of  the  information  contents  which  a  navigation  system  provides,  the 
effectiveness  with  which  this  information  is  presented  will  depend  upon  display 
format  and  consequently  display  cost.  That  is  not  to  say  that  the  optional  display 
will  necessarily  be  the  most  costly.  Determining  this  relationship  was  the  primary 
objective  of  the  miniexperiment. 

The  miniexperiment  evaluated  three  unique  display  technologies  that  were 
selected  based  upon  their  estimated  cost  and  potential  for  actual  shipboard  use. 
Low,  moderate  and  high  cost  categories  were  compiled  based  upon  the  required 
visual  display  capabilities,  computer  characteristics,  and  the  effort  for  computer 
program  development.  Front  end  electronics  such  as  the  navigation  receiver  and 
filter  processor  were  considered  constant  across  all  displays  and  were  not  included  in 
the  cost  tradeoff. 

The  lowest  cost  display  was  represented  as  an  all  DIGITAL  readout  of  navigation 
parameters  such  as  shown  in  Figure  2.  Actual  applications  of  this  type  display  might 
use  light  emitting  diodes  (LED),  liquid  crystal  displays  (LCD),  plasma  or  gas 
discharge  panels,  or  multiple-projection  readouts.  The  device  could  be  small  enough 
to  be  hand  held. 


Figure  2.  Example  of  Digital  Display 


II 


The  moderate  and  high  priced  systems  would  both  require  a  mass  data  display 
with  graphic  capabilities,  probably  a  CRT.  Two  display  concepts,  a  GRAPHIC 
aisalay  or  plan  view  similar  to  traditional  radar  presentations  (see  Figure  3)  and  a 
PERSPECTIVE  display  portraying  the  perspective  scene  as  viewed  out  the  forward 
bridge  windows  (see  Figure  4)  were  considered  moderately  priced.  The  high  priced 
display  system  was  a  STEERING  display  requiring  extensive  high  speed  processing 
and  massive  data  storage  (see  Figure  5).  This  STEERING  display  represented  a 
somewhat  advanced  concept  in  navigation  displays  since  it  integrated  position 
information  with  a  computed  track  prediction  based  upon  hydrodynamic  algorithms 
and  status  inputs  from  ownship. 

In  consideration  of  the  many  design  alternatives  resulting  from  each  of  the  cost 
categories,  it  was  decided  to  evaluate  through  simulation  different  display  variables 
in  each  of  the  DIGITAL,  GRAPHIC  and  PERSPECTIVE  displays.  In  order  to  examine 
as  many  variables  as  possible,  only  an  abbreviated  12-minute  segment  of  the  project 
scenario  was  simulated.  This  segment,  however,  included  the  35-degree  bend  which 
was  considered  to  be  the  most  difficult  maneuver  within  the  waterway. 

Format  variables  examined  in  the  miniexperiment  are  listed  in  Table  3.  From 
this,  five  of  the  most  operationally  effective  formats  were  selected  for  a  full  length 
simulation  in  the  following,  RA-1,  experiment.  Based  upon  conclusions  of  previous 

research'*’^’0  the  STEERING  display  was  considered  adequately  effective  to  be 
recommended  for  testing  in  the  full-length  RA-1  simulation  without  being  examined 
in  the  miniexperiment. 

A  completely  detailed  description  of  the  miniexperiment  including  the  design  of 
all  displays,  the  experimental  design,  scenario  and  performance  measures,  and  all 

results  and  conclusions  are  presented  in  the  interim  report.7 


4 

R.  B.  Cooper,  W.  R.  Bertsche,  and  C.  J.  McCue.  "Simulator  Evaluation  of  Predictor 
Steering,  Short  Range  Collision  Avoidance  and  Navigation  Displays,  Phase  III,  the 
Advanced  Bridge  Design  Program."  Washington,  D.  C.,  U.  S.  Maritime  Administra¬ 
tion,  November  1979. 

^Kockums  Automation  AB.  "Precise  Maneuvering  in  Confined  Waters,  Controlled 
Radial  Steering."  Unpublished.  Maimo,  Sweden. 

^W.  B.  Van  Berlekom.  "Simulator  Investigation  of  Predictor  Steering  Systems  for 
Ships."  In  Transactions  of  Royal  Institute  of  Naval  Architects.  Paper  2,  1977. 

7R.  B.  Cooper  and  K.  L.  Marino.  "Simulator  Evaluation  of  Electronic  Radio  Aids  to 
Navigation  Displays,  The  Miniexperiment."  Washington,  D.C.,  U.S.  Coast  Guard, 
September  1980. 
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TABLE  3.  MINIEXPERIMENT  DISPLAY  VARIABLES 


DIGITAL  DISPLAY  (low  cost) 

Crosstrack  distance  from  centerline 

Crosstrack  velocity 

Course  error 

Heading  to  steer 

Turn  rate 

Recommended  turn  rate 
Distance  to  leadline 
Time  to  leadline 


GRAPHIC  DISPLAY  (moderate  cost) 

Track-up,  true  motion 
Track-up,  relative  motion 
Head-up,  relative  motion 
Ship  heading  vector 
Ship  course  vector 
Scaled  ship  image 
Symbolic  (cross)  ship  image 


PERSPECTIVE  DISPLAY  (moderate  cost) 

60  degree  field  of  view 
90  degree  field  of  view 


2.2  THE  RA-1  EXPERIMENT 

The  objective  of  the  RA-1  experiment  was  to  evaluate,  through  a  full-length 
simulation  of  restricted  waterway  pilotage,  the  effectiveness  of  five  display  formats 
recommended  by  the  miniexperiment  and  two  predictor  steering  displays  recom¬ 
mended  by  other  research.  The  full-length  scenario  employed  in  the  RA-1 
experiment  was  very  similar  to  the  RA-2  scenario  described  in  detail  in  section  3.3 
of  this  report.  It  differed  from  the  miniexperiment  scenario,  however,  in  that  it 
started  and  ended  2.3  nm  from  the  bend  instead  of  0.75  nm.  Also,  ownship  was 
initially  off-set  92  feet  to  the  right  of  the  channel  centerline  necessitating  the  pilot 
to  maneuver  within  the  straight  1^  to  return  to  the  centerline.  In  the  additional 
length  beyond  the  bend,  the  pilot  was  required  to  steady-up  and  maintain  a  track  in 
varying  wind  conditions  and  a  gradually  decreasing  crosscurrent.  Trackkeeping  and 
maneuvering  performance  were  judged  on  how  well  the  subject  initially  returned  to 
the  centerline,  steadied  up  on  it,  maintained  the  centerline  through  the  entrance  leg, 
negotiated  the  turn,  steadied  up  beyond  the  bend,  and  maintained  the  centerline 
through  the  exit  leg.  Speed  control  and  use  of  the  rudder  were  also  analyzed. 
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Format  variables  examined  in  the  RA-1  experiment  are  categorized  by  cost. 
(See  Table  4.)  Results  and  conclusions  of  the  RA-1  experiment  along  with  a 
detailed  description  of  the  displays,  experimental  design,  scenario  and  performance 

E 

measures  are  presented  in  the  RA-1  interim  report.  The  most  important  result, 
however,  was  a  recommendation  based  on  the  simulation  that  the  true  motion, 
track-up  GRAPHIC  displays  and  the  STEERING  display  were  the  most  effective  for 
pilotage  of  the  waterway  as  it  was  simulated. 


TABLE  4.  COST  CATEGORIES  OF  RA-1  DISPLAYS 
Representing  low  cost  systems  (less  than  $500) 

•  Digital  display  (alphanumeric  only)  indicating  crosstrack  distance,  crosstrack 
speed,  and  distance  to  waypoint. 

Representing  moderate  cost  systems  ($5QQ-$5,QQQ) 

•  Digital  display  indicating  crosstrack  distance,  crosstrack  speed,  distance  to 
waypoint,  turn  rate  and  recommended  turn  rate. 

•  Graphic  display  (PPI  type  presentation)  indicating  true  motion  in  a  track-up 
orientation,  and  ownship's  image  with  a  heading  vector. 

•  Graphic  display  indicating  true  motion  in  a  track-up  orientation,  and 
ownship's  image  with  a  course  vector  (direction  of  ship  motion). 

•  Perspective  display  (as  viewed  out  the  forward  windows)  indicating  ownship's 
bow  and  channel  boundary  lines  with  a  90  degree  field  of  view. 

•  Simplified  predictor  steering  display  (PPI  type  presentation  indicating  true 
motion  in  a  track-up  orientation,  and  projection  of  ownship's  track  computed  from 
speed  and  present  rate  of  turn. 

Representing  high  cost  systems  (more  than  $5,000) 

•  Predictor  steering  display  indicating  true  motion  in  a  track-up  orientation, 
and  a  projection  of  ownship's  track  based  upon  the  computed  effects  of  ship 
hydrodynamics,  existing  ship  motion  and  the  amount  of  rudder  applied. 


Based  on  additional  factors  such  as  pilot  preference,  development  cost,  and 
potential  accuracy,  the  true  motion,  track-up  GRAPHIC  display  with  an  ownship 
leading  vector  was  recommended  as  the  benchmark  display  for  RA-2  evaluation. 


7 

R.  B.  Cooper,  K.  L.  Marino,  and  W.  R.  Bertsche.  "Simulator  Evaluation  of  Electronic 
Radio  Aids  to  Navigation  Displays,  The  RA-l  Experiment."  Washington.  D.  C., 
L'mted  States  Coast  Guard,  January  19S1. 
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2.2.1  The  Noise  and  Tracker  Model 

Having  defined  a  benchmark  display  whose  operating  characteristics  and  format 
were  judged  most  effective  with  perfect  position  information,  the  research  next 
endeavored  to  evaluate  the  display  under  real  world  conditions  of  noise  and  system 
limitations.  This  was  accomplished  by  developing  and  testing  a  model  of  an  ALPHA- 
BETA  a  -  b  mathematical  filter  which  would  be  placed  between  the  traditional 
navigation  receiver  and  the  display  input. 

Through  mathematical  processes  this  filter  would  average  random  position  errors 
from  the  receiver  and  provide  a  computed  best-estimate  of  owrtship's  position  on  the 
display.  Naturally,  the  greater  the  noise  level  at  the  receiver,  the  larger  the  random 
error  into  the  filter.  For  a  given  noise  level,  the  amount  of  random  error  can  be 
calculated,  thereby  transforming  the  degree  of  noise  level  directly  into  distance  root 
mean  square  (rms)  about  the  ship's  actual  position. 

This  definition  of  noise  formed  the  basis  for  selection  of  one  of  the  variables  in 
the  RA-2  experiment.  Any  navigation  system,  hyperbolic  line,  time  differential  or 
other,  will  incur  some  error  as  a  result  of  noise.  Since  RA-2  was  intended  to 
evaluate  the  benchmark  display  for  a  generic  navigation  system  and  not  a  specific 
one  (i.e.,  LORAN  C,  Omega,  satellite,  etc.),  noise  errors  were  included  which  are 
representative  of  all  potential  navigation  system  candidates. 

The  evaluation  program  presupposes  the  utilization  of  state-of-the-art  tracking 
filters  where  high  receiver,  processor  and  display  technology  exist.  For  the  RA-2 
experiment  a  critically  damped  a  -  b  filter  with  selectable  rise  time  was  modeled. 
While  not  a  highly  sophisticated  or  optimized  filter  such  as  the  Kalman,  the  a  -  b 
tracker  did  enable  an  examination  of  the  display  effects  caused  by  different  filter 
characteristics  and  the  initiation  of  major  maneuvers. 

During  the  development  of  the  tracker  model,  six  different  tracker  rise  times 
were  simulated.  The  shorter  filter  rise  times  tended  to  give  larger  variations 
between  each  best-estimate  position,  subsequently  producing  a  display  filter. 
Computation  times  were  short,  however,  with  very  little  lag  between  when  the 
position  was  received  and  when  it  was  displayed.  Longer  filter  rise  times,  on  the 
other  hand,  tended  to  present  a  smooth  series  of  best-estimates  of  position.  Here, 
however,  lag  times  were  long,  showing  ownship  considerably  behind  its  actual 
position  in  cases  of  long  rise  times. 

In  the  transit  of  a  straight  leg  or  channel  this  lag  could  be  tolerated  in  favor  of 
the  smoother  best-estimates.  At  a  bend  or  other  maneuver,  however,  the  long  rise 
times  could  be  deleterious.  To  retain  the  advantages  of  long  filter  rise  time,  yet 
reduce  the  consequences  of  lag  during  a  maneuver,  inputs  from  the  gyro  compass  can 
be  used.  These  inputs  would  signal  that  a  maneuver  has  been  initiated  and  that  old 
position  data  used  for  the  best-estimate  should  be  discounted.  While  this  is  an 
extreme  oversimplification  of  gyro-aiding,  it  does  illustrate  the  basic  effect. 
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Verification  of  the  noise  and  tracker  models  were  accomplished  by  steering, 
under  computer  control,  a  30,000  dwt  tanker  through  a  35-degree  turn  at  8  knots! 
Displayed  position  was  then  plotted  versus  the  ship's  actual  position  every  3  seconds. 
Tracker  rise  times  of  3,  6,  12,  24,  42  and  54  seconds  were  evaluated  both  with  and 
without  gyro-aidingin  each  2,  16,  32,  and  64  meter  rms  noise  condition.  These  plots 
as  well  as  a  detailed  description  of  the  models,  their  development,  and  their 

verification  is  found  in  Appendices  A  and  B  of  the  RA-1  Interim  Report.9  Table  5 
summarizes  its  conclusions. 


TABLE  5.  CONCLUSIONS  OF  THE  NOISE  AND 
FILTER  MODEL  DEVELOPMENT 

•  The  optimal  a  -  B  tracker  rise  time  was  shown  to  vary  between  2  and  20 
seconds,  depending  on  the  level  of  rms  noise  added  to  the  signal. 

•  Rise  times  of  3  to  6  seconds  seem  appropriate  for  low  rms  noise  (2  to  16 
meters). 

•  Higher  rise  times  of  10  to  20  seconds  seem  appropriate  for  higher  rms  noise 
(32  to  64  meters). 

•  Additionally,  this  shift  occurs  principally  as  a  function  of  the  noise  masking 
the  tracker  lag  error  for  larger  noise  levels  and  lower  rise  times. 

•  For  very  long  rise  times,  the  errors  asymptotically  approach  the  tracker 
performance  with  2-meter  noise  level.  Thus,  in  the  limit,  the  errors  caused 
by  tracker  lag  dominate  the  noise  errors. 

•  For  moderate  to  large  noise  levels  without  gyro-aiding,  course  errors  can 
become  quite  large  (i.e.,  in  excess  of  15  degrees).  Such  errors  could  severely 
degrade  the  effectiveness  of  a  display  which  employs  course  error  informa¬ 
tion. 

•  With  gyro-aiding  of  the  tracker,  there  is  a  dramatic  reduction  in  tracker 
errors  through  the  turn  which  results  in  a  reduced  maximum  crosstrack  error, 
a  reduction  in  the  maximum  course  error,  and  a  reduction  in  rms  crosstrack 
error. 

•  Rise  times  which  achieve  minimum  errors  fall  between  20  and  42  seconds, 
compared  to  2  to  20  seconds  for  the  unaided  trackers.  The  minimum  values 
are  less  sensitive  to  the  value  of  rise  time  as  a  function  of  noise  level.  Rise 
times  of  24  to  36  seconds  seem  to  be  a  good  choice  for  signal  noise  2  meters 
to  64  meters. 

•  A  major  improvement  with  gyro-aiding  appears  to  be  reduction  of  the 
maximum  course  error.  Such  a  reduction  may  make  feasible  the  use  of 
displays  which  depict  course  and/or  velocity  information. 
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2.3  THE  RA-2  EXPERIMENT 


3ased  upon  display  recommendations  of  the  previous  experiments  and  specific 
operational  and  environmental  considerations  revealed  during  development  of  the 
noise  and  tracker  models,  an  experimental  design  for  the  conduct  of  the  RA-2 
evaluation  was  proposed.  The  simulation  scenario  was  identical  to  the  RA-1 
experiment  except  that  ownship  started  about  one  nautical-  mile  closer  to  the  bend 
making  each  run  only  35  minutes  long.  This  provided  each  subject  time  for  eight 
runs,  and  allowed  a  post  run  interview  to  elicit  his  perception  of  the  run  and  the 
display's  usefulness.  All  runs  were  conducted  using  the  benchmark  display. 

Based  upon  recommendations  of  the  previous  experiments  and  rationale  discussed 
in  Section  3  of  this  report,  the  following  variables  were  simulated: 

Noise  Conditions 

-  16  meter  rms  noise 
32  meter  rms  noise 

Tracker  Characteristics 

-  Tracker  with  3  second  rise  time 

-  Tracker  with  12  second  rise  time 

-  Tracker  with  24  second  rise  time 

-  Gyro-aided  tracker  with  24  second  rise  time 

Subject  selection,  experimental  design  and  performance  measures  were  compar¬ 
able  to  the  RA-1  experiment.  Some  performance  measures  which  were  shown  in  the 
RA-1  experiment  to  be  ineffectual  for  examining  pilotage  behavior  were  omitted. 

Preliminary  observations  of  track  plots  showing  ownships'  tracks  by  variable 
indicated  that  when  displays  with  long  lags  (i.e.,  long  rise  times)  were  used,  subjects 
tended  to  execute  the  bend  with  considerably  less  overshoot.  To  fully  analyze  this 
observation,  plots  were  also  made  of  ownship's  position  as  it  was  shown  on  the 
display.  By  comparing  the  displayed  position  plots  with  actual  position  plots  it  was 
possible  to  ascertain  what  type  of  error  (lag  or  jitter)  and  how  much  error  was  in 
effect  at  each  instant  throughout  the  maneuver. 

Details  of  the  entire  RA-2  experiment  are  contained  within  this  report  as  are  its 
conclusions  and  recommendations.  While  each  are  of  particular  interest  in  the 
context  of  the  overall  program,  there  are  several  conclusions  which  correlate  with 
the  findings  of  APL's  paralleling  PILOT  program.  Specifically,  there  are 
statistically  significant  indications  that  learning  effect  is  a  major  factor  both  in  the 
usefulness  and  acceptance  of  an  electronic  radio  aids  to  navigation  display.  It 
follows,  that  if  formal  familiarization  of  training  in  the  use  of  the  display  is 
provided,  this  learning  effect  will  be  greatly  accelerated.  Also,  the  use  of  a  familiar 
display  format,  specifically  the  plan  view  or  GRAPHIC  presentation,  is  most  readily 
accepted  as  a  navigating  aid  by  pilots.  Noise  which  translates  to  jitter  becomes 
more  obvious  to  the  observer  as  the  update  rate  of  the  display  increases.  The 
effects  of  update  rate  were  not  examined  in  this  experiment. 


Results  of  the  RA-2  experiment  suggest  that  the  human  individual  is  an  excellent 
filter  of  system  error  providing  one  is  able  to  see  the  individual  data  points  (i.e., 
best-estimate  positions);  but  not  at  a  fast,  distracting  rate.  Finally,  considering  the 
many  experimental  runs  and,  in  some  cases,  repeated  use  of  the  GRAPHIC  display  by 
the  same  subjects,  there  was  no  series  of  runs  in  poor  visibility  which  equaled  or 
surpassed  in  performance  the  best  runs  which  had  been  made  under  visual  conditions 

(i.e.,  the  AN-VISUAL  experiments.) 

Figure  6  compares  piloting  performance  of  the  best  (most  accurate)  radio  aid 
display  to  the  best  (gated  buoys  marking  both  sides  of  the  channel)  visual  buoy 
configuration.  Performance  differences  are  most  obvious  in  the  turn.  The  radio  aid 
display  resulted  in  a  wide  dispersion  of  tracks  and  an  overshoot  of  the  bend,  while 
the  visual  scenario  resulted  in  controlled  tracks  close  to  the  center  of  the  channel. 
The  differences  in  performance  are  due  to  varying  certainty  as  to  when  to  initiate 
the  turn  maneuver  and  how  to  control  the  turning  maneuver  prior  to  pull  out.  Turn 
maneuver  actions  appear  more  certain  with  visual  information  and  less  certain  (i.e., 
more  variable)  with  the  graphic  radio  aid  display. 

While  significant  contributions  have  been  made  by  this  program  toward 
understanding  the  effects  on  pilotage  of  various  navigation  displays  and  operating 
characteristics,  the  fact  remains  that  in  poor  visibility  or  when  all  visual  aids  have 
been  removed,  the  electronic  radio  aids  to  navigation  display  as  evaluated  does  not 
produce  pilotage  performance  comparable  to  visual  navigation. 


°M.W.  Smith  and  W.  R.  Bertsche.  "Aids  to  Navigation  Principal  Findings  on  the 
CAORF  Experiment.  The  Performance  of  Visual  Aids  to  Navigation  as  Evaluated  by 
Simulation."  U.S.  Coast  Guard  Office  of  Research  and  Development,  DOT-CG- 
835285-A,  Washington,  February  1981. 

11  VI.  W.  Smith  and  W.  R.  Bertsche.  "Aids  to  Navigation  Principal  Findings  Report  on 
the  Channel  Width  Experiment:  The  Effects  of  Channel  Width  and  Related  Variables 
on  Piloting  Performance."  Washington,  D.C.,  U.S.  Coast  Guard,  January  19S1. 


Section  3 

EXPERIMENTAL  DESIGN 


The  RA-2  experiment  was  intended  to  evaluate,  through  man-in-the-loop 
simulation,  pilots'  trackkeeping  and  shiphandling  performance  with  a  GRAPHIC  radio 
aids  to  navigation  display  under  conditions  of  simulated  system  noise  and  lag  errors 
introduced  by  the  tracker  filter  system  itself.  This  experiment  was  conducted 

12 

similarly  to  the  RA-1  experiment  described  by  Cooper  et  al,  1980.  In  conjunction 

with  an  earlier  miniexperiment  ^  the  experiment  selected  a  true  motion,  track-up 
GRAPHIC  display  as  the  benchmark  display  for  evaluation  in  eight  combinations  of 
noise,  filter  bandwidth,  and  filter  aiding  techniques  .  The  RA-1  experiment 
recommended  the  GRAPHIC  display  with  either  heading  or  course  vectors. 
Subsequently,  the  decision  was  made  to  employ  the  heading  vector  display  since 
estimated  course  would  be  severely  degraded  by  the  presence  of  signal  noise. 
Logistics  of  the  experiment  precluded  the  evaluation  of  both  heading  and  course 
vector  displays  with  ail  variables. 

The  experimental  variables  were  selected  as  a  result  of  the  RA-1  development 
and  derivation  of  performance  data  for  the  critically  damped  ALPHA-BETA  (-^-3) 
position  tracker.  Conclusions  of  this  development  and  rationale  for  the  selection  of 
variables  are  presented  in  the  following  section. 

The  RA-2  experiment  was  conducted  in  a  full-length  scenario,  using  the  same 
waterway  and  environmental  characteristics  (i.e.,  wind  and  current)  as  the  RA-1 
experiment.  Similar  data  collection  and  analysis  techniques  were  also  employed. 

Instructions  to  subjects  were  taken  from  the  RA-1  experiment.  Due  to  the 
seemingly  repetitious  nature  of  the  RA-2  experiment  (i.e.,  RA-1  used  different 
displays  and  RA-2  used  only  one),  it  was  necessary  to  provide  continuous  motivation 
to  the  subjects  to  perfect  their  pilotage.  This  was  accomplished  by  eliciting  from 
each  subject  how  well  he  performed  in  each  previous  run  and  then  encouraging  him 
to  "do  better."  While  this  may  appear  somewhat  artificial,  previous  experience  had 
shown  that  when  runs  are  repetitious  and  similar,  pilots  tend  to  try  new  techniques 
or  strategies  to  overcome  boredom.  Since  no  visual  feedback  was  provided  other 
than  what  the  subjects  saw  on  the  display,  the  critique  of  their  perception  of  own 
performance  compared  to  their  actual  performance  was  a  good  indication  of  how 
effectively  the  display  information  was  presented. 
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3.1  EXPERIMENTAL  VARIABLES 


The  selection  of  variables  for  the  RA-2  experiment  was  based  on  tradeoffs 
between  noise  and  tracker  performance  in  consideration  of  real  world  characteris¬ 
tics  and  potential  system  implementation.  The  earlier  research,  which  examined 
tracker  rise  times  of  3,  6,  12,  24,  42,  and  54  seconds  with  noise  of  from  2  meters  (6.6 
feet)  rms  to  64  meters  (211.2  feet)  rms,  resulted  in  the  following  conclusions: 

1.  The  2-meter  (6.6-foot)  rms  noise  level  is  probably  too  close  to  the  RA-1 
baseline  (zero  rms  noise)  for  any  performance  comparison.  Additionally,  the  2- 
meter  error  probably  is  accurate  enough  not  to  even  warrant  filtering  in  the  first 
place. 


2.  The  64-meter  (211.2-ioot)  rms  noise  level  is  considerably  greater  than 
presently  exists  in  state-of-the-art  navigation  systems.  As  a  result,  its  evaluation 
wouid  serve  only  to  define  the  performance  on  an  obsolete  system  parameter. 

3.  Both  the  16-meter  (52.8-foot)  and  the  32-meter  (105.6-foot)  rms  noise  levels 
were  proposed  as  variables  for  the  RA-2  experiment.  Each  represents  state-of-the- 
art  accuracy  of  certain  navigation  systems  as  well  as  hypothesized  extremes  of 
performance  which  would  benefit  from  the  incorporation  of  an  a- 3  tracker. 

4.  An  S-meter  (26.3-foot)  rms  noise  level  will  be  tested  during  the  presimulation 
phase  of  the  experiment  along  with  the  16-meter  noise  to  determine  if  there  is  an 
incremental  difference  in  performance  between  the  two.  If  there  is,  the  8-meter 
noise  would  be  substituted  for  the  16-meter  noise  in  the  RA-2  experiment  since  it 
more  closely  represents  potential  navigation  system  capabilities. 

5.  a  -  3  filters  with  rise  times  of  3,  12,  and  24  seconds  were  selected  to 
represent  the  spectrum  of  filter  characteristics.  These  rise  times  were  selected 
because  they  demonstrated  an  important  range  of  errors  when  used  in  16-  and  32- 
meter  noise  but  without  gyro-aiding.  The  3-second  rise  time  results  in  large  errors 
which  are  principally  the  result  of  signal  noise.  The  24-second  rise  time  results  in 
large  errors  nearly  equivalent  to  those  for  the  3-second  rise  time,  but  they  are 
principally  the  result  of  filter  lag  errors.  The  12-second  rise  time  represents  the 
nearly  minimum  error  condition. 

6.  The  ct-8  filter  with  gyro-aiding  would  be  tested  using  only  a  24-second 
rise  time.  Gyro-aiding  is  considered  less  essential  at  slower  rise  times  because  lag  is 
minimal.  The  experiment  proposes  to  determine  if  gyro-aiding  ensures  increased 
system  effectiveness  at  24  seconds.  The  24-second  a- 8  filter  with  gyro-aiding 
vould  be  evaluated  in  both  16-  and  32-meter  rms  noise  levels. 

As  a  result,  two  levels  of  noise,  three  levels  of  tracker  rise  time  and  two  levels 
of  filter  aiding  were  selected  for  the  RA-2  experiment.  They  are  shown  in  Figure  7 
Plots  of  the  offline  verification  of  performance  for  each  of  these  variables  are 
presented  in  Appendix  A. 


•MAY  BE  REPLACED  BY  8-METER  RMS  NOISE  AS  A  RESULT  OF 
PRESIMULATION  FINDINGS. 


Figure  7.  Experimental  Design  Matrix 


3.2  SIMULATOR  FACILITY 


A  comprehensive  description  of  the  simulator  facility  used  in  the  conduct  of  the 
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RA-1  experiment  is  provided  in  section  2.2  of  Cooper  and  Marino.  Briefly,  the 
simulator  is  a  fully  equipped  ship's  bridge  with  a  visual  simulation  capability.  It  has 
been  developed  in  conjunction  with  the  U.  S.  Coast  Guard  to  support  their  conduct  of 
aids  to  navigation  research.  Simulated  radar  and  navigation  displays  are  driven  by  a 
Digital  Equipment  Corporation  GT-44  computer  graphics  system  with  PD  P-1 1/40 
central  processor  and  VT-11  graphic  generation  hardware.  The  computer  CRT 
display  is  mounted  in  a  free-standing  pedestal  and  equipped  with  required 
controls/indicators  and  bearing  rings  to  simulate  a  PPI  type  bridge  display  or  various 
radio  aids  to  navigation  formats.  Other  computers  control  both  the  electronic 
bridge  display  and  visual  system  although  visuals  were  not  used  during  the  radio  aids 
evaluation.  The  computer  program  reflects  ownship  characteristics,  maneuverabil¬ 
ity,  hydrodynamic  influences,  and  individual  scenario  (i.e.,  waterway  and  environ¬ 
ment)  conditions.  In  the  case  of  the  RA-2  experiment,  it  also  modeled  navigation 
system  n.  'se  and  system  filter  characteristics  for  the  display  evaluation.  The 
computer  acility  provides  a  continuous  automatic  recording  of  ship  position,  ship 
status,  ana  bridge  control  manipulations  for  post  simulation  data  reduction,  graphic 
and  statistical  analysis. 


3.3  SIMULATION  SCENARIO 

The  RA-2  scenario  differed  from  the  RA-1  experiment  only  in  the  location  of  the 
initiation  point  which,  for  RA-2,  was  1.2  nautical  miles  from  the  bend. 

3.3.1  Ownship  Characteristics 

The  ship  was  similar  in  all  characteristics  to  the  one  run  during  both  previous 

radio  aids  to  navigation  experiments  and  the  AN-CAORF^  and  AN-VISUAL^ 
experiments. 

Ownship:  29,694  dwt  tanker  ballasted 

34.6  foot  draft 
Depth  below  keel- 1  foot 
Height  of  eye-45  feet 
Wheelhouse  midships 
84  foot  beam 
595  foot  length 

Initial  speed:  6.5  knots  through  the  water  (S  knots  ground  speed) 

Initial  heading:  341°T  gyro 
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^M.  W.  Smith  and  '-V.  R.  Bertsche,  oo.  cit,  January  1981. 


3.3.2  Operating  Area 

The  scenario  waterway  and  starting  position  of  ownship  is  shown  in  Figure  g 
This  waterway  was  also  used  in  the  abbreviated  scenario  of  the  miniexperiment  and 
the  RA-1  experiment.  It  is  comparable  to  the  AN-CAORF  scenarios  17  through 

24,  *7  and  AN- VISUAL  scenarios  IS,  20,  22,  and  24 18.  Unlike  these  visual 
experiments,  there  are  no  buoys  shown  on  the  radio  aids  to  navigation  display. 

The  waterway  was  a  500  foot  wide  channel  with  a  35-degree  left  bend. 
Environmental  effects  were  simulated  as  follows: 

For  entire  scenario: 

Current:  1.5  knot  flow  to  341  degrees  true  decreasing  to  0 

knots 

From  161  degrees  true  variable  13  percent 

30  knots  plus/minus  10  percent  variable  per  600- 
second  period  plus  gust  at  10  percent  velocity  per 
60-second  period. 

Ownship  originated  approximately  92  feet  to  right  of  the  channel  centerline,  1.2 
nautical  miles  south  of  the  bend.  Instructions  to  the  subject  were  to  return  to  the 
centerline  and  maintain  the  center  of  the  channel  as  much  as  possible  through  the 
transit.  There  was  no  traffic  and  no  visual  scene  (i.e.,  poor  visibility  conditions  are 
simulated). 

3.4  RADIO  AIDS  TO  NAVIGATION  SYSTEM  MODEL 

Radio  navigation  systems  are  presently  being  implemented  to  facilitate  ship 
piloting  in  narrow  waterways.  Such  systems  include  typically  a  radio  receiver,  a 
signal  processing  unit  or  filter,  and  a  position  display  device.  Given  state-of-the-art 
electronics,  most  systems  are  now  microcomputer  based  and  utilize  digital  filtering 
techniques.  The  basic  system  elements  are  diagrammed  in  Figure  9.  The  simulation 
model  represents  the  receiver  and  the  signed  processing  unit  as  single  trackers  (see 
Figure  10).  Experimental  results  and  filter  analyses  of  this  experiment  should  thus 
not  be  wholly  attributed  to  either  the  receiver  or  the  signal  processing  unit. 

A  two-axis  radio  signal  system  has  been  assumed  for  the  implementation:  north- 
south  signal  and  an  east-west  signal.  The  noise  in  these  signals  is  assumed  to  be 
independent  over  the  sampie  interval  chosen.  A  white  noise  source  with  a  Gausian 
distribution  is  assumed. 


17M.  W.  Smith  and  W.  R.  Bertsche,  op.  cit,  February  1981. 
^M.  W.  Smith  and  W.  R.  Bertsche,  op.  cit,  February  1981. 


Wind  direction: 
Wind  velocity: 
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Figure  3.  Scenario  Waterway 
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Figure  9.  Basic  Elements  of  a  Radio  Navigation  System 


Figure  1C.  Representation  of  a  Radio  Navigation 
System  with  a  -  3  Trackers 


The  navigation  system  is  implemented  as  shown  in  Figure  11.  Note  that  the  x 
coordinate  is  positive  in  the  north  direction  and  the  y  coordinate  is  positive  in  the 
east  direction.  All  subsequent  derivations  refer  to  this  coordinate  system.  Identical 
values  for  alpha  are  used  in  the  x  and  y  coordinate  trackers.  Critical  damping  is 

assumed  for  both  trackers,  S  =  (  ct^)/(2  -a  )•  Identical  equations  were  implemented 
in  each  tracker  with  the  appropriate  changes  to  notation  (x  and  y).  A  detailed 
description  of  the  a  -S  tracker  characteristics  and  equations  are  presented  in 
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Appendix  A  of  the  RA-1  report. 

Performance  of  the  navigation  system  was  evaluated  during  the  RA-1  develop¬ 
mental  phase,  using  the  system  to  track  a  30,000  dwt  tanker  through  a  35-degree 
turn  at  8  knots.  Full  ship  hydrodynamic  equations  were  utilized  to  represent  the  ship 
response.  This  ship  is  identical  to  that  used  in  the  RA-1  experiment  and  the 
miniexperiment.  A  simple  autopilot  was  utilized  for  executing  the  turn  to  achieve 
repeatability. 


5R  =  2  (heading  error) 


where: 

6r  =  rudder  angle 


This  autopilot  exhibited  approximately  a  10  percent  overshoot  in  heading  for  the  35- 
degree  turn.  Typically,  this  implementation  caused  a  hard  over  rudder  to  be  applied 
for  most  of  the  turn,  resulting  in  a  high  turn  rate.  Such  a  response  presents  the 
trackers  with  the  greatest  possible  transient  for  the  given  ship,  turn  angle,  and 
initial  ship's  speed. 

The  data  in  Figure  12  show  a  typical  response  of  the  trackers  to  the  35-degree 
turn.  Such  plots  allow  a  quick  visual  analysis  of  the  tracker  performance.  As 
emphasized  by  the  dashed  line,  an  oscillatory  response  is  evident  in  the  turn.  This 
might  properly  be  traced  to  the  nonlinear  variations  in  ship's  north  and  east 
velocities  through  the  turn.  It  is  interesting  to  compare  performance  before  and 
after  the  turn.  Prior  to  the  turn  the  trackers  seem  to  have  settled  to  rather  a 
smooth  response.  Following  the  turn,  the  trackers  seem  to  be  continually  perturbed 
almost  in  steady  state  oscillation. 
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Figure  li.  Implementation  of  Navigation  5; 
for  Experimentation 
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Figure  12.  T/pical  Plot  oi  Tracker  Response  to  35-Degree  Turn 
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Implementation  of  the  gyro-aiding  filter  was  made  'sfiown  in  Figure  13. 
Equations  were  implemented  for  both  the  north  and  east  signal  data.  An  a  -  6 
tracker  was  added  to  filter  the  gyro  signal.  Its  rise  time  is  independently  controlled 
from  the  position  trackers.  The  alpha  value  for  this  tracker  was  denoted  as  ct^.  As 
noted  in  the  figure,the  current  estimates  of  ownship's  heading,  turn  rate,  and  north 
and  east  velocities  are  used  to  calculate  the  velocity  changes.  These  estimated 
changes  are  used  to  improve  the  estimates  of  ship's  position  and  velocities.  The 
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applicable  equivalents  are  described  in  the  RA-1  report. 

The  RA-1  analysis  of  gyro-aiding  sought  to  select  an  appropriate  rise  time  for 
the  gyro  tracker.  The  baseline  conditions  selected  were  32-meter  rms  signal  noise, 
with  a  30-second  rise  time  in  the  position  trackers  (  a  =  0.027).  The  sample  period 
was  equal  to  1  second.  The  results  implied  that  a  rise  time  of  between  2  and  4 
seconds  for  the  gyro  tracker  achieves  good  performance.  A  gyro  tracker  rise  time 
of  3  seconds  was  chosen  for  all  subsequent  gyro-aiding  analyses  (c^  =  0.19). 

A  second  analysis  effort  sought  to  optimize  the  multiple  of  turn  rate  in  the  speed 
change  equations.  Evaluation  runs  with  zero  rms  noise  were  run,  and  maximum 
crosstrack  error  and  maximum  course  error  were  evaluated.  A  multiple  between 
0.85  and  1.0  seems  to  minimize  errors.  The  multiple  of  0.85  was  selected  for  all 
subsequent  gyro-aiding  analyses. 

3.5  SUBJECT  SELECTION 

Eight  licensed  pilots  from  local  and  mid-Atlantic  pilots'  associations  acted  as 
subjects  for  the  experiment.  Runs  were  completed  within  8  hours  to  minimize 
fatigue  effects.  This  included  time  for  an  explanation  of  the  display's  operation  as 
well  as  brief  postrun  interviews.  Ail  subjects  selected  were  familiar  with  the 
response  characteristics  of  a  30,000  dwt  tanker.  Nevertheless,  they  were  given  an 
opportunity  to  maneuver  the  simulated  ship  during  a  preexperiment  familiarization 
run.  Every  effort  was  made  to  enlist  the  participation  of  those  pilots  who  were 
subjects  in  the  RA-1  and  miniexperiment.  This  substantially  minimized  the  display 
familiarization  requirement.  In  addition  to  critiquing  their  own  performance 
following  each  run,  subjects  were  encouraged  to  voice  their  opinions  and  recom¬ 
mendations  both  for  display  design  and  in  regard  to  the  overall  simulation. 

3.6  ADMINISTRATION 
3.6.1  Experimental  Design 

The  simulator  experiment  was  intended  as  an  evaluation  of  overall  operational 
effectiveness  and  of  potential  user  acceptance  of  the  GRAPHIC  radio  aids  to 
navigation  display  in  a  systematically  varied  noise  and  filter  design  environment. 

Organization  of  the  variables  was  designed  to  investigate  within-subject  effects, 
thus  minimizing  individual  differences  and  encouraging  a  higher  probability  of 
finding  significance  than  would  be  experienced  by  between-subject  effects.  This,  of 
course,  meant  a  large  number  of  scenario  repetitions  for  each  subject  with  the 
resultant  possibility  of  introducing  order  effect  (e.g.,  learning,  anticipation, 
boredom,  etc.,  caused  by  repetition).  Tests  for  learning  effect  were  conducted  on 
all  data  and  the  results  are  presented. 
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Figure  13.  Implementation  of  Navigation  System  with  Gyro-Aiding 


3.6.2  Assignment  Schedule 


The  administration  schedule  is  shown  in  Table  6.  Each  subject  received  all 
variables  described  in  Figure  6,  which  were  counterbalanced  between  subjects  to 
minimize  the  order  effect.  Two  noise  levels  (16  and  32  meters  rm s)  were  arranged 
so  that  each  appeared  before  the  other  an  equal  number  of  times.  The  3-,  12-,  and 
24-  second  unaided  tracker  rise  times  and  the  one  24-second  gyro-aided  rise  time 
variables  were  arranged  so  that  every  one  followed  each  of  the  others  an  equal 
number  of  times. 

In  summary,  the  experimental  design  permitted  the  best  mix  of  administration 
options  to  perform  the  experiment  inexpensively  and  expediently  yet  enabling  it  to 
retain  the  necessary  requirements  to  ensure  statistical  validity  and  adequate  levels 
of  confidence. 


TABLE  6.  SUBJECT  ASSIGNMENTS 


Order  of 
Administration 

1 

2 

3 

Subject 

4 

Number 

5 

6 

7 

S 

1st 

16/3* 

32/3 

16/24G 

32/24G 

16/12 

32/12 

16/24 

32/24 

2nd 

32/12 

16/12 

32/24 

16/24 

32/24G 

16/24G 

32/3 

16/3 

3rd 

16/24 

32/24 

16/12 

32/12 

16/3 

32/3 

16/24G 

32/24G 

4th 

32/24G 

16/24G 

32/3 

16/3 

32/24 

16/24 

32/12 

16/12 

5th 

32/3 

16/3 

32/24G 

16/24G 

32/12 

16/12 

32/24 

16/24 

6th 

16/12 

32/12 

16/24 

32/24 

16/24G 

32/24G 

16/3 

32/3 

7th 

32/24 

16/24 

32/12 

16/12 

32/3 

16/3 

32/24G 

16/24G 

8th 

16/24G 

32/24G 

16/3 

32/3 

16/24 

32/24 

16/12 

32/12 

•The  first  number  indicates  rms  noise  in  meters,  the  second  number  indicates  tracker  rise 
time  in  seconds,  and  "G"  indicates  the  tracker  is  GYRO  aided  (see  Figure  2). 


Section  4 

DATA  COLLECTION  AND  ANALYSES 


Data  collection  and  analysis  of  performance  for  the  RA-2  experiment  was 
conducted  similar  to  the  RA-1  experiment.  Methods  of  data  retrieval,  storage  and 

compilation  are  detailed  in  die  RA-12*  and  miniexperiment22  reports.  Instead  of 
graphically  illustrating  entrance  and  exit  leg  performance  separately  as  had  been 
done  in  RA-1,  however,  the  RA-2  analysis  connected  both  legs  together  to  show 
performance  through  the  entire  run.  This  feature  enabled  the  analyst  (and 
subsequently  the  reader  of  the  report)  to  subjectively  compare  performance  between 
variables  simply  by  glancing  between  pages.  Statistical  analyses  were,  of  course, 
employed  to  indicate  where  significant  differences  existed. 

4.1  TRACKKEEPING  ANALYSIS 

The  trackkeeping  analysis  of  runs  performed  during  the  simulation  was  conducted 
by  combining  all  runs  within  each  variable,  then  computing  and  plotting  the  mean 
track  and  standard  deviation  of  tracks  for  the  variable.  An  example  of  the  resultant 
plots  is  shown  in  Figure  14.  Note  that  this  example  shows  only  Leg  1.  The  RA-2 
plots  show  both  legs,  the  entrance  (Leg  I)  and  exit  (Leg  2),  connected  together.  The 
horizontal  axis  in  the  graphs  represent  discrete  along  channel  positions  at  equal  475- 
foot  intervals.  These  intervals,  called  "data  lines,"  originate  at  the  waypoint  or 
center  of  the  bend  on  the  RA-2  graphs.  In  other  words,  data  line  zero  occurs 
midway  through  the  bend. 

The  performance  measure  plotted  on  the  upper  graph  in  Figure  14  is  the  trace  of 
the  mean  across  channel  position  of  the  ship’s  center  of  gravity  averaged  at  each 
data  line.  The  vertical  axis  of  this  graph  represents  across  channel  distance  in  feet. 
The  starboard  channel  boundary  is  at  0,  the  channel  centerline  is  the  dashed  line 
plotted  at  250,  and  the  port  channel  boundary  is  at  500. 

The  vertical  axis  in  the  center  graph  is  an  absolute  scaler  quantity  in  feet.  The 
solid  plotted  line  represents  the  standard  deviation  of  the  ship’s  center  of  gravity  for 
all  transits  calculated  at  each  data  line. 

The  vertical  axis  of  the  lower  graph  is  cross  channel  distance  in  feet.  The  dashed 
horizontal  line  at  500  is  the  port  channel  boundary.  The  dashed  line  at  0  is  the 
starboard  channel  boundary.  The  information  contained  in  the  crosshatched  plot  is  a 
modified-com bination  of  that  shown  in  the  upper  two  plots.  The  center  solid  line 
inside  the  crosshatched  area  is  the  mean  track  line  of  the  center  of  gravity.  The 
upper  and  lower  lines  bounding  the  crosshatched  area  are  each  two  times  the 
standard  deviation  away  from  the  mean  center  of  gravity  line.  These  lines  define 
the  envelope  which  would  contain  95  percent  of  all  ship  center  of  gravity  tracks  in 
the  population  of  transits  under  the  specific  experimental  condition  given  the  actual 
sample  transits  performed  by  the  subjects. 
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Ibid. 
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.  B.  Cooper  and  K.  L.  Marino,  op.  cit. 


Example  of  Summary  Plot  Analysis 


Appendix  E  contains  the  statistical  analysis  of  track  plots  which  were  used  to 
identify  significant  differences  in  trackkeeping  performance  as  a  result  of  the 
experimental  variables.  The  figures  were  constructed  as  a  result  of  statistically 
comparing,  using  the  F-statistic  at  the  p  ^  0.10  level  of  significance,  one  standard 
deviation  of  crosstrack  variability  between  conditions  at  each  data  line.  These 

figures  illustrate  the  difference  in  variability  which  existed  along  the  tracklines  for 
each  variable,  and  where  it  occurred.  The  data  are  used  as  statistical  support  for 
conclusions  about  the  group  mean  track  and  crosstrack  variability  plots  discussed  in 
Section  4. 

Additional  plots  were  drawn  which  represented  ownship's  position  as  it  was 
presented  on  the  display.  These  plots  were  also  drawn  as  mean,  standard  deviation 
and  combined  tracks  so  that  they  could  be  compared  with  the  actual  positions  of  the 
ship  in  each  variable.  While  these  plots  are  not  published,  they  were  helpful  in 
enabling  the  analyst  to  determine  whether  differences  in  trackkeeping  performance 
were  due  to  perceptual  problems  or  the  fact  that  the  display  was  exhibiting  a  large 
position  error. 

4.2  MANEUVERING  PERFORMANCE 

The  maneuvering  analysis  was  conducted  for  two  major  maneuvers,  the  transit  of 
a  straight  leg  with  wind  and  current  astern,  and  the  transit  of  a  straight  leg  with 
wind  and  current  from  the  port  quarter.  The  initial  maneuver  was  to  return  to  the 
centerline  from  a  position  92  feet  to  the  right  of  centerline.  The  second,  was  to 
negotiate  a  35-degree  left  bend  in  the  channel.  To  analyze  maneuvering 
performance  for  the  different  variables,  two  analytical  techniques  were  employed. 
The  first  was  an  analysis  of  the  overall  distribution  and  frequency  of  rudder 
commands,  course  commands  and  engine  orders.  The  distribution  of  these  control 
activities  is  illustrated  graphically  on  the  same  page  as  the  trackkeeping  plots  and  in 
relatively  the  same  format.  All  control  activities  were  recorded  at  the  time  they 
actually  occurred  and  were  subsequently  plotted  at  the  position  (crosstrack  as  well 
as  along  track)  which  ownship  occupied  when  they  occurred.  This  resulted  in  a  tool 
which  enabled  the  analyst  to  determine  not  only  what  control  activities  were  used, 
but  in  many  cases  why  they  were  used. 

The  second  technique  was  a  statistical  analysis  of  the  quantitative  maneuvering 
data.  A  more  detailed  description  of  the  individual  measures  which  were  analyzed  is 
presented  in  Appendix  D.  Essentially  they  consisted  of  the  following: 

1.  Return  to  and  steady  up  on  the  channel  centerline  following  the  initial  offset 

(wind  and  current  astern) 

a.  Along  track  distance  required  to  return  to  the  centerline 

b.  Crosstrack  overshoot  of  the  centerline  following  the  return  to  it 

c.  Along  track  distance  required  to  steady  up  on  the  centerline  of  the 
entrance  leg 

d.  Following  negotiation  of  the  bend  (wind  and  current  port  quarter)  —  along 
track  distance  required  to  steady  up  on  the  centerline  of  the  exit  leg 


V 


2.  Initial  turn  rudder  applied  before  the  bend 

a.  Along  track  distance  before  the  bend  when  the  initial  rudder  was  applied 

b.  Magnitude  of  the  initial  turn  rudder 

c.  Maximum  initial  turn  rudder  that  was  applied 

d.  Frequency  of  turn  rudders  which  were  applied 

3.  Check  rudder  applied  beyond  the  bend 

a.  Along  track  distance  beyond  the  bend  when  the  initial  check  rudder  was 
applied 

b.  Magnitude  of  the  initial  check  rudder 

Statistical  differences  as  determined  from  the  t-statistic  at  the  90  percent  level 
of  significance  (p  ^0.10)  are  indicated  on  the  tables  in  which  the  measures  are 
presented. 

4.3  USER  ACCEPTANCE 

Structured  interviews  were  administered  by  the  test  director  at  the  end  of  each 
run.  This  enabled  subjects  to  respond  to  questions  about  the  run  while  it  was  fresh  in 
their  minds.  The  interviews  were  structured  by  use  of  questionnaires  to  be  filled  out 
jointly  by  the  test  director  and  subject.  This  ensured  completeness  and  enabled  the 
test  director  to  clarify  any  questions  which  arose. 

The  questionnaires  were  brief  (did  not  exceed  five  minutes  for  normal 
administration)  and  primarily  addressed  perceived  individual  performance  (self- 
appraisal). 

Questions  were  developed  and  arranged  so  that  the  subject  could  first  give  an 
appraisal  of  his  performance  in  each  segment  of  the  waterway  (i.e.,  poor,  fair,  good, 
or  excellent);  then  he  could  choose  from  a  number  of  potential  deficiencies  (e.g., 
rudder  too  early  or  rudder  too  late)  to  adequately  describe  his  difficulty.  The 
questionnaire  is  presented  in  Appendix  C. 


SECTION  5 

RESULTS  AND  CONCLUSIONS 


Results  and  conclusions  of  the  RA-2  experiment  are  based  upon  both  descriptive 
and  statistical  analysis  of  pilotage  performance.  As  in  the  previous  RA-l  and 
miniexperiment,  trackkeeping  and  maneuvering  performance  as  well  as  user 
acceptance  are  compared  between  experimental  variables.  In  the  case  of  the  RA-2 
experiment,  these  variables  were  system  noise  and  tracker  filter  design 
characteristics. 

To  properly  evaluate  both  the  independent  effects  of  these  variables  and  their 
interaction  effects,  the  analysis  was  conducted  at  two  levels.  First,  all  runs  were 
divided  into  two  groups;  one  in  which  16  meter  rms  noise  was  simulated  and  one  in 
which  32  meter  rms  noise  was  simulated.  These  two  groups  each  with  a  sample  size 
of  32  were  graphically  and  statistically  compared  to  determine  the  effect  of  the 
different  noise  levels  on  pilotage  performance.  Ail  noise  conditions  were  then 
combined  and  runs  were  divided  into  four  groups  by  type  of  tracker.  Comparisons  in 
pilotage  performance  between  these  four  groups  determined  the  effect  of  filter  rise 
times  on  pilotage  performance.  Sample  sizes  were  16  for  each  group. 

The  next  analysis  examined  interaction  effects.  It  specifically  addressed  the 
effect  on  oilotage  performance  of  gyro  aiding  the  tracker  system.  This  analysis 
compared  the  unaided  tracker  with  24-second  rise  time  against  the  gyro-aided 
tracker  with  24-second  rise  time  in  each  of  the  two,  16  meter  rms  and  32  meter  rms, 
noise  conditions.  A  final  analysis  compared  pilotage  performance  of  all  subjects' 
initial  runs  with  their  final  runs  to  identify  the  effect  of  learning  on  the  overall 
experiment. 

Of  the  64  runs  which  were  conducted  during  the  RA-2  experiment,  none  produced 
effects  considered  serious  enough  to  invalidate  a  run.  Several  excursions  from  the 
channel  occurred  as  a  result  of  overshooting  or  undershooting  the  bend.  These,  it 
will  be  shown,  were  attributed  to  the  combination  of  display  design  and  tracker 
design,  not  system  noise.  In  general,  the  shiphandling  characteristics  of  a  relatively 
large,  slow  ship  in  a  confined  waterway  and  the  environmental  effects  (i.e.,  wind  and 
current)  were  considered  by  all  subjects  to  be  difficult  but  not  unrealistic.  Likewise, 
all  subjects  agreed  the  displayed  information  was  sufficient  to  accomplish  the 
pilotage  in  the  given  poor  visibility;  but  they  would  never  attempt  such  a  transit 
without  first  "proving"  the  display  during  good  visibility. 

5.1  EFFECT  OF  NOISE  ON  PILOTAGE  PERFORMANCE 

The  overall  effect  of  noise  alone  on  the  radio  aids  to  navigation  display  as  it  was 
simulated  is  best  illustrated  in  Figures  15,  16,  and  17.  These  plots  show  very  little 
if  any  difference  in  trackkeeping  performance  as  a  result  of  noise  alone.  This 
conclusion  is  also  supported  in  the  statistical  analysis  of  trackkeeping  consistency, 
Figure  E-l  of  Appendix  E.  Further,  Tables  7  and  8  which  quantitatively  describe  the 
pilots'  maneuvering  performance  show  no  significant  differences  for  any  of  the 
measures  analyzed. 

It  is  the  conclusion  of  this  analysis  that  within  the  limits  of  noise  which  were 
simulated,  and  given  all  the  display  design  characteristics  present  in  the  experiment, 
the  pilot  himself  was  a  very  adaptable  filter.  He  was  able  to  achieve  good  pilotage 
performance  relatively  independent  of  the  effects  of  noise  to  which, be  .was  exposed. 

■*  "*  ■*>  tL  - 


38 


Trackers  in  16  Meter  RMS  Noise 


Trackers  in  32  Meter  RMS  Noise 


TABLE  7.  EFFECT  OF  NOISE  ON  MANEUVERING 
PERFORMANCE  WITH  ALL  TRACKERS 


TABLE  S.  MEAN  FREQUENCY  OF  CONTROL  ACTIVITIES 


EXPERIMENTAL  VARIABLES 

RUDDER 

COMMANDS 

COURSE 

COMMANDS 

ENGINE 

ORDERS 

Effect  of  Noise  with  All  Trackers 

All  trackers  in 

16  meter  RMS  noise 

21 

11 

3 

-  32  meter  RMS  noise 

Effect  of  Tracker  Design 

20 

14 

2 

In  all  noise  conditions 

tracker  with  3-second  rise  time 

21 

12 

2 

tracker  with  12-second  rise  time 

22 

11 

3 

tracker  with  24-second  rise  time 

21 

13 

2 

-  gyro-aided  tracker 

18 

13 

3 

Tracker  with  3-second  rise  time  in 

16  meter  RMS  noise 

26 

10 

2 

-  32  meter  RMS  noise 

16 

14 

2 

Tracker  with  12-second  rise  time  in 

-  16  meter  RMS  noise 

19 

10 

i 

-  32  meter  RMS  noise 

26 

12 

2 

Tracker  with  24-second  rise  time  in 

16  meter  RMS  noise 

23 

10 

2 

32  meter  RMS  noise 

19 

15 

2 

Gyro-aided  tracker  in 

16  meter  RMS  noise 

18 

12 

3 

32  meter  RMS  noise 

18 

15 

2 

Effect  of  Learning 

Subjects'  initial  run 

30* 

11 

3 

Subjects'  final  run 

♦StAriiTirallv  Hifforanr  at  th>»  n  10  level  of  significanc 

16 

_ 

13 

4 
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Because  the  analysis  describes  the  average  performance  of  ail  trackers  under 
different  noise  conditions;  the  "better"  trackers  could  have  compensated  for 
deleterious  effects  of  "poorer"  ones.  For  this  reason  the  next  analysis  examined  the 
effects  of  individual  tracker  design  in  all  noise  conditions. 

5.2  EFFECT  OF  FILTER  RISE  TIME  ON  PILOTAGE  PERFORMANCE 

Obvious  effects  on  pilotage  performance,  both  in  trackkeeping  and  maneuvering, 
were  in  evidence  as  the  result  of  different  tracker  rise  times  and  gyro  aiding 
technique.  These  effects  are  shown  graphically  in  Figures  18  through  22,  but  are 
most  apparent  in  the  statistical  comparison  of  maneuvering  performance.  Table  9. 
When  combining  both  noise  levels  (i.e.,  equal  samples  of  16  and  32  meter  rms  noise) 
as  shown  in  Figure  1?,  there  was  little  visible  difference  in  trackkeeping 
performance  among  the  unaided  trackers,  but  a  tendency  to  overshoot  the  bend  with 
the  gyro-aided  tracker.  Table  9  and  Figures  :  S',  2C,  and  11 ,  also  show  there  were  no 
significant  differences  in  distance  to  return  to  the  centerline  or  to  steady  up  when 
noise  conditions  were  combined. 

Ownship  jitter  was  most  obvious  with  the  short  rise  time  trackers.  It  required 
pilots  to  consciously  do  their  own  filtering  by  estimating  actual  position  somewhere 
between  each  consecutively  displayed  position.  Results  of  the  overall  track  plots 
indicate  that  this  "visual  filtering"  was  successful. 

As  tracker  rise  times  lengthened,  two  additional  factors  began  to  influence  the 
display.  First,  tracker  positioning  on  the  display  became  smoother,  making  ownship 
jitter  less  obvious.  As  a  result,  visual  filtering  became  easier.  Second,  the  longer 
rise  time  introduced  lag  into  the  system  which  introduced  a  bias  error  as  soon  as 
ownship  began  to  maneuver.  In  other  words,  given  a  tracker  with  a  lengthy  rise 
time,  jitter  would  be  minimized,  but  ownship’s  along  track  and  crosstrack  position 
could  be  displayed  with  a  large  error  depending  upon  ownship’s  speed  and/or  rate  of 
turn. 

The  net  effect  on  pilots’  behavior  resulting  from  the  tradeoff  between  display 
jitter  and  Jag  are  revealed  in  the  analysis  of  pilotage  performance  with  the  different 
tracker  designs.  This  comparison  of  performance  between  trackers  with  different 
rise  times  suggests  that  pilots  are  capable  of  "visual  filtering"  when  the  need  to  do 
so  is  obvious.  However,  in  the  absence  of  feedback  as  to  ownship’s  actual  position  in 
the  waterway,  the  existence  of  display  lag  and  its  effects  on  their  pilotage  will  go 
undetected.  What  pilots  saw  on  the  display  and  the  way  the  display  portrayed  their 
pilotage  was  the  only  feedback  the  pilots  received.  When  a  large  bias  error  was 
present  on  the  display  as  a  result  of  filter  lag,  the  pilots  had  no  knowledge  of  it  and 
maneuvered  the  ship  as  though  it  was  presenting  "perfect"  position  information. 
Unlike  display  jitter,  the  pilots  could  not  compensate  for  lag. 

For  this  reason,  the  effects  of  filter  lag  on  pilotage  performance  were  found  to 
be  large  compared  to  the  effects  of  jitter.  This  is  best  illustrated  by  a  hypothetical 
scenario  of  a  ship  being  piloted  with  a  relatively  long  rise  time  filter.  The  scenario 
could  be  similar  to  that  experienced  by  pilots  when  they  used  the  unaided  tracker 
with  24-second  rise  time  in  32  meter  rms  noise  to  execute  a  35-degree  turn.  The 
scenario  is  plotted  in  Figure  A-6  of  Appendix  A  which  shows  ownship’s  actual  track 
and  the  position  of  ownship  as  it  was  presented  on  the  display  every  second.  The 
magnitude  of  display  error,  its  direction  from  ownship’s  actual  position  and  upon 
what  occasion  the  error  occurred  is  essential  to  understanding  the  pilots'  behavior. 
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Figure  19.  Tracker  wilh  3-Second  Rise  Time  in  All  Noise  Conditions 


Tracker  with  24-Second  Rise  Time  in  All  Noise  Condi  lions 


Tracker  in  All  Noise  Conditions 


TAliLL  9.  FFFliCT  OF  TRACKFR  OFSIGN  ON  MANIil  JVliRING 
1‘liRFORMANCt  IN  ALL  NOISII  CONDITIONS 
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Hypothctical-^cefiarfo 

The  pilot  is  on  a  straight  leg  approaching  a  sharp  35-degree  bend  at  a 
relatively  slow  speed  of  6  knots.  The  long  rise  time  tracker  has 
stabilized  from  all  previous  maneuvers  and  is  producing  only  slight  jitter 
as  a  result  of  the  well  filtered  noise.  The  errors  indicated  in  the  jitter 
are  random  all  around  ow nshipts  actual  position.  The  pilot  is  doing  his 
own  " visual  filtering"  between  jitters  to  estimate  actual  position  of 
ownship.  Lag  errors  are  almost  non-existent. 

As  soon  as  the  pilot  initiates  his  turn  maneuver  and  a  rate  of  turn 
develops,  the  effect  of  lag  begins  to  dominate.  Since  ownship  is 
traveling  slowly  but  the  turn  rate  develops  rapidly,  the  lag  error 
transforms  more  into  a  crosstrack  error  than  an  along  track  error.  Some 
along  track  lag,  however,  does  exist.  (See  Figure  A-6  in  Appendix  A.)  As 
the  turn  rate  of  ownship  increases  and  ownship  actually  begins  to  proceed 
around  the  bend,  the  lag  error  which  is  biased  toward  the  outside  of  the 
turn  also  increases.  The  display  thus  indicates  to  the  pilot  that  he  is  not 
moving  into  the  bend  as  rapidly  as  he  actually  is,  and,  as  a  result,  the 
pilot  applies  additional  rudder  to  even  further  tighten  his  turn. 

The  pilot's  perception  of  this  maneuver  is  that  ownship  was  extremely 
slow  in  responding  to  his  initial  turn  rudder.  If  he  had  preconceived  ideas 
of  how  the  ship  should  handle  whether  from  previous  simulation  runs  or 
prior  operational  knowledge,  he  would  be  concerned  about  this  inconsis¬ 
tency  and  would  assume  the  ship  will  respond  equally  sluggishly  to  the 
check  rudder. 

Once  ownship's  turn  rate  is  established  and  ownship  is  well  into  the 
bend,  the  lag  error  transforms  even  more  into  a  crosstrack  bias  than 
before.  This  causes  the  display  to  show  ownship  as  overshooting  the  new 
centerline  and  elicits  additional  measures,  usually  hard  rudder,  to  be 
applied  by  the  pilot. 

As  shown  in  the  figure,  the  filter  does  not  restabilize  until  well  into 
the  exit  leg  beyond  the  bend.  This  means  that  if  ownship  were  actually 
on  the  centerline,  for  the  beginning  of  the  leg  it  would  be  presented  on 
the  display  well  to  the  right  of  center.  Gradually,  as  the  lag  error 
reduces,  ownship  on  the  display  would  move  toward  the  center.  Pilots, 
without  any  external  source  of  reference,  would  have  no  way  of  knowing 
this  and  would  attempt  to  keep  the  displayed  ownship  in  the  center  of  the 
channel. 

The  result  of  this  scenario,  then,  is  that  if  a  pilot  were  able  to 
maintain  ownship  on  the  centerline  of  the  display,  he  would  surely 
undershoot  the  bend  and  end  up  to  the  left  of  center  well  into  the  exit 
leg. 


Given  the  above  scenario,  it  is  possible  to  see  the  effects  a  long  rise  time  tracker 
with  long  lag  could  have  on  pilotage  performance.  While  these  effects  did  occur  to 
some  degree  in  the  RA-2  experiment,  there  is  evidence  to  support  the  conclusion 
that  they  were  not  severely  detrimental  to  the  pilotage;  and,  in  fact,  appeared  to 
improve  it.  An  explanation  for  this  is  derived  from  the  results  of  the  previous  RA-1 
and  miniexperiment.  In  none  of  these  experiments  which  used  displays  witfr;-.*  $ 
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"perfect"  position  information  was  pilotage  performance  better  or  even  comparable 

to  visual  runs  (i.e.,  AN-CAORF  and  AN-VISUAL  experiments).^’  ^  In  ail  cases 
the  displays  led  to  at  least  some  overshoot  of  the  centerline,  and  occasionally  an 
overshoot  of  the  channel  boundary.  This  effect,  however,  was  minimal  with  the 
GRAPHIC  and  STEERING  displays,  and  was  one  of  the  bases  for  selecting  the 

GRAPHIC  as  the  benchmark  display 

The  fact  remains  that  there  were  tendencies  of  pilots  to  overshoot  the  bend  when 
they  used  the  benchmark  display  with  "perfect"  position  information.  If,  as  is 
suggested  by  the  hypothetical  scenario,  the  effect  of  tracker  lag  causes  pilots  to 
undershoot  the  bend,  then  a  combination  of  the  benchmark  display  with  tracker  lag 
should  provide  for  "centerline"  trackkeepirg  performance. 

This  irony,  of  course,  is  not  the  solution  to  either  of  the  problems.  In  fact,  it 
would  probably  not  have  occurred  had  pilots  been  more  familiar  with  the  use  of  a 
GRAPHIC  display  for  piloting  or  had  the  scenario  been  different.  For  example,  with 
a  fast  ship,  the  tracker  lag  would  probably  be  transformed  more  into  an  along  track 
error  than  a  crosstrack  error.  The  result  would  be  that  the  actual  ship  would  lead 
the  displayed  ship  into  the  bend  and  the  possibility  of  overshoot  would  be  immense. 

5.3  EFFECT  OF  GYRO-AIDING  ON  PILOTAGE  PERFORMANCE 

The  potentially  detrimental  effects  of  tracker  lag  are  well  acknowledged  Dy 
developers  of  tracker  filters  for  navigation  systems.  As  a  result,  some  filters  have 
been  designed  to  "know"  when  an  intentional  maneuver  is  initiated,  thus  modifying 
the  filter  operation  and  eliminating  much  of  the  lag.  Although  there  are  several 
methods  for  signaling  the  initiation  of  a  maneuver  to  the  filter, the  RA-2 
experiments  used  a  gyrocompass  input.  The  results  on  display  accuracy  are  shown  in 
Figure  A-8,  Appendix  A.  Note  that  while  noise  error  remains  in  the  form  of  ownship 
jitter,  once  the  maneuver  is  initiated  there  is  little  indication  of  bias  error  due  to 
lag. 

In  summary,  when  pilots  use  the  gyro-aided  tracker,  they  see  the  same  amount  of 
jitter  as  on  the  unaided  displays,  but  experience  none  of  the  effects  of  lag. 
Assuming,  as  this  analysis  has  concluded,  that  pilots  are  able  to  "visually  filter"  out 
the  jitter,  then  performance  with  the  gyro-aided  tracker  should  be  very  comparable 
to  performance  with  die  "perfect"  position  GRAPHIC  display  in  the  RA-1 
experiment. 
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Trackkeeping  performance  with  the  gyro-aided  tracker,  Figures  18  and  22, 
indicates  the  same  moderate  overshoot  of  the  centerline  and  high  crosstrack 

variability  experienced  in  the  RA-1  experiment.26  It  also  suggests  a  somewhat  less 
than  favorable  performance  compared  to  the  unaided  trackers.  The  probable  cause 
for  this  has  been  discussed.  Further  evidence  to  support  the  discussion  is  presented 
in  Table  9.  Note  that  for  all  trackers  the  average  initial  turn  rudder  was  applied  at 
approximately  the  same  distance  before  the  turn  (0.154  to  0.162  nm),  and 
approximately  the  same  magnitude  of  initial  turn  rudder  (30  to  35  degrees)  was  used 
between  the  unaided  trackers  and  the  gyro-aided  trackers.  However,  in  comparing 
subsequent  rudders  (i.e.,  those  applied  after  the  initial  turn  rudder),  pilots  used 
significantly  more  rudder  with  the  unaided  tracker  (35  degrees)  than  the  gyro-aided 
one  (25  degrees).  In  other  words,  as  suggested  in  the  hypothetical  scenario,  pilots 
perceived  themselves  as  entering  the  bend  equally  well  with  both  trackers.  Once 
into  the  turn,  however,  the  unaided  tracker  made  them  perceive  that  they  were 
going  to  overshoot  the  bend  and  they  applied  additional  turn  rudder.  With  the  gyro- 
aided  display  they  proceeded  normally,  maintaining  much  of  the  original  rudder  with 
which  they  originally  initiated  the  turn. 

This  analysis  describes  the  effects  of  each  tracker  design  in  the  combined  noise 
environment.  The  next  analysis  was  conducted  to  identify  the  potential  interactive 
effects  of  each  tracker  design  with  each  noise  level. 

5.4  INTERACTIVE  EFFECTS  OF  TRACKER  DESIGNS  AND  NOISE  LEVELS 

Based  upon  the  findings  described  in  sections  5.2  and  5.3,  the  analysis  of 
trackkeeping  and  maneuvering  performance  was  interpreted  using  the  following 
rationale: 

1.  An  expedient  return  to,  and  steady  up  on  the  channel  centerline  following  any 
maneuver  was  considered  normal  pilotage  performance. 

2.  An  overshoot  of  the  exit  leg  centerline,  but  not  an  excursion  from  the  channel 
was  considered  normal  pilotage  performance. 

3.  An  undershoot  of  the  bend  was  considered  to  result  from  tracker  lag  in  the 
presentation  of  ownship's  position  on  the  display. 

4.  High  pilot  anxiety  or  excessive  rudder  application  in  the  turn  maneuver  was 
considered  to  result  from  the  effect  of  iag. 

In  light  of  this  rationale,  a  review  of  Appendix  E,  which  is  the  statistical  analysis 
of  trackkeeping  performance,  illustrates  the  effects  of  the  various  tracker  designs 
and  noise  levels.  In  each  figure,  the  upper  plot  shows  mean  track  of  all  runs  in  each 
condition.  The  lower  plot  shows  the  variability  of  all  runs  as  one  standard  deviation 
of  crosstrack  distance. 


Figure  E-2  shows  no  difference  in  mean  tracks  between  3-  and  12-second  rise 
time  trackers  in  the  low  (16  meter  rms)  noise.  Those  differences  in  variability  which 
are  shown  cannot  be  attributed  to  noise  since  the  tracker  with  3-second  rise  time 
showed  the  most  jitter  and  in  fact,  has  the  significantly  lower  variability.  The  plot 
suggests  that  with  jitter  on  the  display  pilots  were  conservative  in  their  maneuvers 
to  return  to  and  steady  up  on  the  centerline.  Figure  E-3  shows  the  same  two 
trackers  in  the  higher  (32  meter  rms)  noise.  Here  the  tracker  with  3-second  rise 
time  (i.e.,  the  most  jitter)  shows  a  significantly  more  gradual  return  to  the- 
centerline  and  steady-up  in  the  exit  leg.  There  is  relatively  little  difference  in 
variability. 

In  the  comparison  between  trackers  with  12-  and  24-second  rise  times  (Figures  E- 
4  and  E-5),  significant  difficulty  is  indicated  in  steadying  up  with  the  24-second 
tracker  at  both  noise  levels.  Here, the  tracker  with  minimal  jitter  (i.e.,  the  24- 
second  tracker)  produced  minimal  crosstrack  variability. 

Figures  E-6  and  E-7  compare  the  effects  of  large  jitter  from  a  3-second  tracker 
with  minimal  jitter  but  long  lag  from  a  24-second  tracker.  In  the  lower  (16  meter 
rms)  noise  where  there  is  less  jitter,  the  3-second  tracker  permitted  a  relatively  fast 
return  to  the  centerline  beyond  the  bend.  In  the  higher  32  meter  rms  noise,  this 
return  to  the  centerline  was  much  more  gradual.  With  the  24-second  tracker, 
performance  was  again  affected  by  the  difference  in  noise.  In  the  lower  noise, 
steadying  up  appeared  to  be  difficult  (see  Figure  E-6).  The  pilots  saw  no  jitter  on 
their  display,  but  were  deceived  by  the  lag  effect.  At  the  higher  noise  level  (see 
Figure  E-7)  both  the  3-second  and  24-second  filters  showed  comparable  tracks  with  a 
very  gradual  return  to  and  steady  up  on  the  new  centerline. 

Figures  E-8  and  E-9  show  the  trackkeeping  performance  between  24-second 
trackers  which  were  gyro-aided  and  those  which  were  not.  It  is  obvious  from  both 
figures  that  the  gyro-aided  tracker  produced  a  greater  overshoot  beyond  the  bend 
which  according  to  the  analysis  rationale  is  comparable  to  expected  performance. 
The  figures  also  show  that  with  high  (32  meter  rms)  noise  the  gyro-aided  trackers 
produced  a  mean  track  comparable  to  the  unaided  trackers,  but  with  significantly 
greater  consistency  (i.e.,  less  crosstrack  variability)  among  pilots. 

5.4.1  Trackers  with  3-Second  Rise  Time 

In  order  to  make  recommendations  on  tracker  design  for  use  with  the  benchmark 
display,  a  determination  of  the  causes  for  the  effects  revealed  in  the  analysis  is 
required.  This  is  accomplished  by  comparing  the  pilotage  performance  for  each 
tracker  when  it  was  used  in  each  of  the  two  different  noise  conditions.  The  analysis 
makes  use  of  both  descriptive  track  plots  and  a  statistical  comparison  of 
quantitative  measures. 

Figures  23,  24,  and  25  show  the  trackkeep..ig  performance  which  was  achieved 
when  the  trackers  with  3-second  rise  time  were  used.  Very  similar  trackkeeping 
performance  is  in  evidence  regardless  of  noise.  Table  10  shows  no  difference  in 
maneuvering  attributed  to  noise  level,  although  there  was  a  larger  variability  among 
pilots  in  steadying  up  initially  with  the  higher  noise.  Overall,  it  is  concluded  that 
the  tracker  with  3-second  rise  time  produced  comparable,  satisfactory  pilotage 
performance  in  both  noise  levels.  This  performance  is  attributed  primarily  to  the 
pilots'  "visual  filtering"  ability. 
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Figure  24.  Tracker  with  3-Second  Rise  Time  in  16  Meter  RMS  Noise 


TABLE  10.  EFFECT  OF  TRACKER  WITH  3-SFOOND  RISE  TIME 
ON  MANEUVERING  PERFORMANCE 
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5.4.2  Tracker  with  12- Second  Rise  Time 

Pilotage  performance  as  indicated  in  Figures  26,  27  and  28  again  shows  more 
similar  trackkeeping  between  the  two  noise  levels  when  the  tracker  with  12-second 
rise  time  was  used.  There  was  a  high  variability  among  pilots  in  steadying  up  beyond 
the  bend  in  the  low  noise  condition.  Table  11  shows  a  very  gradual  steady  up  in  the 
high  noise  condition.  Closer  analysis  of  these  measures  revealed  no  particular 
detriment  to  maneuvering  performance  as  a  function  of  these  effects.  As  reported 
in  the  statistical  analysis  of  track  plots,  section  5.4  and  Appendix  E,  the  major 
difference  in  pilotage  performance  between  the  3-  and  12-second  trackers  was 
overall  superior  maneuvering  to  return  to  the  centerline  and  steady  up  with  the  12- 
second  tracker. 

5.4.3  Tracker  with  24-Second  Rise  Time 

Figures  29,  30  and  31  show  the  differences  in  trackkeeping  performance 
resulting  from  noise  for  the  tracker  with  24-second  rise  time.  While  Table  12  shows 
no  differences  in  maneuvering  performance,  differences  are  in  evidence  on  the  mean 
CC  plots  (Figures  30  and  31).  They  indicate  considerable  difficulty  returning  to  the 
centerline  beyond  the  bend  for  both  noise  conditions.  Pilot  variability  also  is  large 
in  this  particular  area  with  the  higher  noise. 

The  conclusion  is  that  the  noise  variable  did  not  significantly  affect  trackkeeping 
performance.  The  introduction  of  lag  bias  as  a  result  of  long  rise  time,  however, 
produced  difficulties  in  maneuvering,  particularly  returning  to  the  centerline  and 
steadying  up. 

5.4.4  Gyro-Aided  Tracker  with  24-Second  Rise  Time 

A  comparison  of  pilotage  performance  between  when  the  gyro-aided  tracker  with 
24-second  rise  time  was  used  in  the  higher  noise  and  when  it  was  used  in  the  lower 
noise  is  illustrated  in  Figures  32,  33  and  34;  and  Table  13.  This  tracker  has  minimal 
lag  and  minimal  jitter,  and  is  most  similar  to  the  "perfect"  position  display  of  the 
RA-i  experiment.  It  is,  therefore,  not  surprising  to  find  that  pilots  overshot  the 
bend  very  much  like  they  did  in  the  RA-1  experiment.  It  is  surprising, to  find  that 
they  experienced  larger  crosstrack  variability  beyond  the  bend  in  the  lower  noise 
condition.  No  other  indications  point  to  noise  level  as  having  an  effect  on  the  gyro- 
aided  tracker's  performance. 

5.5  LEARNING  EFFECTS 

Although  certain  interactions  of  the  experimental  variables  were  shown  to  have 
some  effect  on  pilotage  performance,  none  are  believed  to  have  as  great  an  effect 
on  overall  safety  of  the  transit  as  the  design  of  the  display  format  itself.  Ability  to 
stay  within  the  channel  was  not  impaired  by  noise.  Tracker  lag,  on  the  other  hand, 
went  undetected  by  the  subjects.  It  resulted  in  a  biased  trackkeeping  performance 
which,  because  of  original  inadequacies  in  the  display,  caused  pilotage  performance 
to  appear  better  than  it  actually  was.  Although  ail  pilots  were  familiar  with  the  use 
of  the  benchmark  display,  none  had  ever  been  given  the  opportunity  to  "prove"  its 
use  in  good  visibility.  As  a  result,  the  pilots  were  "blindly"  following  the  display, 
assuming  its  representation  of  ownship  in  the  channel  was  accurate.  This  decision  to 
test  the  benchmark  display  without  allowing  pilots  to  "prove"  its  accuracy  probably 
resulted  in  experimental  results  that  are  less  optimistic  than  would  otherwise  have 
been  achieved. 
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Figure  26-  Effect  of  Noise  on  Tracker  with  12-Second  Rise  Time 
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Figure  27.  Tracker  with  12-Second  Rise  Time  in  16  Meter  RMS  Noise 
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Figure  2S.  Tracker  with  12-Second  Rise  Time  in  32  Meter  RMS  Noise 


Figure  29.  Elfect  of  Noise  on  a  Tracker  with  24-Second  Rise  Time 


TABLE  12.  EFFECT  OF  TRACKER  WITH  2'J-SECOND  RISE  TIME 
ON  MANEUVERING  PERFORMANCE 
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I'igure  32  •  Effect  of  Noise  on  a  Gyro-Aided  Tracker 
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Pilots  participating  in  the  RA-2  experiment  had  used  the  benchmark  display  in 
previous  experiments.  Nevertheless,  they  had  never  used  it  with  jitter  or  lag 
present.  In  order  to  determine  what  effects,  if  any,  learning  to  use  the  display  with 
jitter  and  lag  had  upon  the  experiment,  an  analysis  comparing  subjects'  performance 
during  their  initial  runs  with  performance  in  their  final  runs  was  conducted.  These 
runs  contained  a  balancedmmixture  of  experimental  variables. 

Figures  35,  36  and  37  show  some  improvement  in  trackkeeping  performance 
among  pilots  as  a  function  of  learning  the  display,  waterway,  and  ship  characteris¬ 
tics.  A  review  of  the  bend  (Figure  35)  shows  high  crosstrack  variability  among  pilots 
when  recovering  from  their  maneuvers  in  the  initial  runs.  Final  runs,  on  the  other 
hand,  were  marked  by  bolder  maneuvers  to  return  to  the  channel  centerline 
following  the  initial  maneuver  and  steadying  up  beyond  the  bend.  This  conclusion  is 
illustrated  by  the  mean  CG  tracks  in  Figures  36  and  37 and  supported  statistically  in 
Table  14.  Although  the  bolder  maneuvers  occurred  during  the  final  runs  with  a 
relatively  high  variability  among  pilots,  it  suggests  that  whatever  learning  did  occur 
was  manifested  by  the  unique  behavior  of  individual  pilots,  i.e.,  some  felt  confident 
enough  to  make  bolder  maneuvers,  while  others  chose  to  continue  their  original 
strategies  which  had  proven  successful.  Table  8  shows  significantly  fewer  rudder 
commands  during  the  final  runs  than  the  initial  runs.  This  is  a  characteristic 
behavior  of  pilots  which  results  when  they  become  confident  of  the  operation. 

5.6  User  Acceptance  of  the  System 

The  appraisal  of  how  potential  users  would  accept  the  electronic  radio  aids  to 
navigation  display  was  based  on  (1)  experimenter  observations,  (2)  informal  subject 
comments  during  the  experiment,  and  (3)  responses  to  a  self-appraisal  questionnaire 
administered  at  the  end  of  each  run.  The  questionnaire  supplied  pilots'  opinions  of 
the  displays  and  an  assessment  of  their  own  performance  in  each  segment  of  the 
waterway.  The  questionnaire  and  resulting  responses  is  presented  in  Appendix  C. 

Table  15  shows  a  summary  of  questionnaire  responses.  In  general,  the  pilots 
believed  they  adapted  to  ail  variations  of  noise  and  tracker  characteristics,  and  they 
were  satisfied  with  their  overall  performance.  The  pilots  stated  that  the  display 
with  the  most  jitter  which  was  the  tracker  with  3-second  rise  time  in  32  meter  rms 
noise,  was  least  accurate.  They  assessed  their  overall  performance  with  it  to  be 
"fair  to  poor."  This  was  the  worst  subjective  rating  for  any  of  the  displays. 

The  pilots  gauged  most  of  their  display  preference  in  the  straight  legs  because 
this  is  where  jitter  was  most  obvious.  In  the  bend,  they  showed  significant 
dissatisfaction  with  their  own  performance  whenever  they  used  a  long  rise  time 
tracker.  The  pilots,  themselves,  had  no  way  of  knowing  that  tracker  lag  was  causing 
their  maneuvering  difficulties.  As  a  result  they  tended  to  blame  themselves  for  the 
maneuvering  difficulty. 

Pilots  stated  that  the  information  update  rate  was  acceptable.  They  believed  a 
consistent  rate  of  about  once  every  second  was  better  than  a  slower  or  inconsistent 
one.  There  was  further  indication  that  if  the  update  rate  were  increased,  with 
conditions  of  large  jitter  there  might  be  a  temptation  to  interpolate  every  update  of 
ownship's  position  and  thus  increase  the  annoyance  factor.  Pilots  were  more 
concerned  abouc  consistency  of  the  update  rate  than  its  absolute  frequency. 
Consistency  allowed  them  to  pace  their  vigilance  and  monitoring  tasks. 
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Figure  35.  Overall  Turning  Performance  as  a  Function  of  Learning 


.  Overall  Trackkeeping  Performance  during 
Subjects'  Initial  Run 


Overall  Trackkeeping  Performance  during 
Subjects'  Final  Run 


Throughout  the  straight  legs,  pilots'  trackkeeping  performance  was  considered  by 
them  to  be  better  when  they  used  displays  with  least  jitter.  These  were  indicated  to 
be  the  trackers  with  12-  and  24-second  rise  time  in  either  noise  condition  and  the  3- 
second  trackers  in  the  lower  level  noise.  With  the  3-second  tracker  in  higher  noise, 
pilots  were  uncertain  of  their  ability  to  maneuver  or  maintain  a  steady  track  on  the 
centerline. 

In  maneuvering  through  the  bend,  pilots  continued  to  be  aware  of  jitter,  mostly 
with  the  3-second  tracker,  but  somewhat  with  the  12-second  tracker  in  the  higher 
noise.  With  the  24-second  tracker,  however,  all  subjects  perceived  extreme 
difficulties  in  shiphandling.  This  difficulty,  unbeknownst  to  them,  was  the  effect  of 
tracker  lag.  As  shown  on  Table  15  initial  maneuvers  with  the  24-second  tracker 
were  perceived  as  worse  than  the  steady-up  maneuvers.  This  was  probably  due  to 
the  filter  stabilizing  by  the  time  steadying  up  was  begun. 

The  major  complaint  was  that  with  the  24-second  tracker  the  ship  initially 
responded  very  sluggishly,  but  then  checked  up  quickly.  The  result  was  that  pilots 
subsequently  took  extreme  measures  to  prevent  what  looked  like  a  large  overshoot 
of  the  bend;  then  blamed  themseives  for  not  starting  with  a  larger  or  earlier  turn 
rudder.  For  the  most  part,  pilots  were  very  displeased  with  their  performance  when 
they  maneuvered  with  the  24-second  trackers,  and  with  the  system's  performance 
when  they  maneuvered  with  the  3-second  trackers.  Ironically,  all  the  way  through 
the  exit  leg,  pilots  were  relatively  satisfied  with  their  performance  regardless  of 
which  display  they  used.  This  suggests  that  the  pilots  did,  in  fact,  adapt  well  to  the 
jitter  and  had  begun  to  treat  it  as  a  normal  characteristic  by  the  end  of  the  run. 

The  conclusion  .is  that  pilots  perceived  themseives  as  being  able  to  handle  both 
jitter  and  the  effects  of  tracker  lag.  That  they  would  fully  "trust"  or  appreciate  a 
display  with  characteristics  similar  to  the  3-  and  24-second  tracker  is  doubtful. 
From  all  indications  of  the  user  acceptance  analysis,  it  appears  that  the  12-second 
tracker  would  be  the  best  of  the  unaided  trackers,  but  that  the  gyro-aided  tracker 
would  be  most  preferred  for  pilotage. 
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TRACKER  PERFORMANCE  BATA  ON  THE  RA-2  VARIABLES 


The  performance  of  the  a  -  8  trackers  was  evaluated  for  a  30,000  dwt  tanker 
executing  a  35-degree  turn  at  8  knots.*  This  appendix  contains  the  individual  track 
plots  from  which  performance  data  was  derived  and  discussed  in  Section  2.  Position 
tracker  rise  times  of  3,  12,  and  24  seconds  are  shown  for  each  condition.  The 
individual  track  plots  appear  in  the  designated  figures.  Tracker  sample  time  was  T  = 
1  second. 


q-fl  Trackers  Without  Gyro  Aiding 

rms  noise  s  16  m  (52.8  ft)  Figures  .A-I  through  A-3 

rms  noise  =  32  m  (105.6  ft)  Figures  A -4  through  A-6 

A  A  A  A 

q-ftTrackers  With  Gvro  Aiding  (Gyro  tracker  rise  time  3  seconds;  ZAXn+1— O.850n+1  T  Vn*in0n+1 
AYn+1«O.850n+1  T  Vncos0n+1 

rms  noise  =  16  m  (52.8  ft)  Figure  A-7 

rms  noise  =  32  m  ( 105.6  ft)  Figure  A-8 


*R.B.  Cooper,  K.L.  Marino,  and  W.R.  Bertsche.  "Simulator  Evaluation  of  Electronic 
Radio  Aids  to  Navigation  Displays,  The  RA-1  Experiment."  Washington,  D.C.,  U.S. 
Coast  Guard,  August  1980. 
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Appendix  B 

INSTRUCTIONS  TO  SUBJECTS 

Introduction.  We  have  asked  you  to  participate  in  this  experiment  to  evaluate  the 
effectiveness  of  a  navigation  display  for  piloting  a  ship  through  a  restricted 
waterway  in  limited  visibility.  You  will  use  the  display  eight  times  to  determine  its 
usefulness  in  navigating  the  ship  for  5  miles  through  a  500-foot  wide  channel.  The 
display  will  be  described  in  detail  when  you  are  ready  to  use  it. 

For  the  next  10  minutes  you  may  familiarize  yourself  with  the  response 
characteristics  of  the  ship  and  the  proficiency  of  your  helmsman.  Using  the 
gyrocompass,  which  is  initially  set  at  341  degrees  true,  we'd  suggest  you  make 
severed  large  course  changes  to  determine  the  amount  of  rudder  needed  to  initiate 
and  check  the  ship's  swing,  and  the  effect  of  rpm  on  turning  maneuvers.  Ownship  is 
a  30,000  dwt  tanker  in  ballast  with  a  28-foot  draft.  It  has  a  595-foot  length  overall, 
and  an  84-foot  beam.  You  will  begin  half  ahead  at  about  7  knots,  with  a  1-1/2  knot 
following  current.  You  may  ask  any  questions  and  try  any  maneuvers  during  this 
time. 

Scenario.  During  this  experiment,  we  will  be  measuring  how  well  you  keep  to  the 
center  of  the  channel.  Your  goal,  therefore,  is  to  keep  on  the  channel  centerline  as 
much  as  possible  from  your  starting  position  up  to  the  second  waypoint  (reference 
chart).  The  channel  is  500  feet  wide  and  36  feet  deep. 

Your  starting  position  will  be  1.2  nm  from  a  35-degree  left  bend  in  the  channel. 
Ownship  is  92  feet  to  the  right  of  the  channel  centerline,  and  you  should  return  to 
the  centerline  as  soon  as  practical.  Your  head  is  steady  on  341  degrees  true,  and  the 
ship's  speed  is  8  knots  over  ground  at  the  beginning  of  the  run.  The  engine  order 
telegraph  (EOT)  is  set  at  half  ahead,  40  rpm,  to  make  6.5  knots  through  the  water. 
RPM  changes  are  permitted;  however,  we  would  like  you  to  maintain  about  8  knots 
overall  transit  speed.  Use  of  speed  variations  are  limited  to  full  ahead,  half  ahead, 
slow  ahead,  dead  slow  ahead,  and  stop.  No  astern  bells  are  available. 

There  will  be  a  following  current  of  1.5  knots  at  the  beginning  of  the  run.  This 
current  will  decrease  steadily  while  approaching  the  turn.  After  the  turn,  the 
current  will  be  3/4  knot  broad  on  the  port  quarter.  It  will  return  gradually  to  aft 
(reference  chart).  There  will  be  a  wind  of  30  knots.  The  wind  direction  is  from  aft 
during  the  first  leg  and  from  broad  on  the  port  quarter  during  the  second  leg. 

Display.  The  display  that  you  will  use  is  a  graphic  display  showing  ownship's  position 
in  the  channel.  The  display  will  be  true  motion  oriented  track-up.  With  the  true 
motion  display,  ownship  comes  on  at  the  bottom  of  the  screen  and  resets  after  it  has 
traveled  three-fourths  of  the  distance  across  the  screen.  In  the  track-up  mode,  the 
picture  comes  on  with  the  channel  centerline  oriented  up,  and  ownship  moves 
through  it.  Once  you  have  completed  the  turn,  the  display  will  automatically  change 
to  the  new  track-up,  and  ownship  will  reset  to  the  bottom  of  the  screen. 

The  display  is  provided  with  a  heading  vector  which  corresponds  to  gyro  heading 
and  is  drawn  to  the  edge  of  the  screen.  Ship  image  is  the  actual  shape  and  size  of 
ownship  scaled  to  the  display.  You  will  be  on  heading  of  341  degrees  in  the  first  leg, 
half  ahead,  1.2  nautical  miles  from  the  turn.  Ownship  is  92  feet  to  the  right  of  the 
channel  centerline.  Please  rpturn  to  the  centerline  as  soon  as  practical. 
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Questionnaire.  At  the  end  of  each  run,  we  will  ask  you  "20  questions"  about  how 
well  you  think  you  piloted  different  segments  of  the  waterway.  As  you  transit  the 
waterway,  you  might  try  to  think  of  how  your  pilotage  could  have  been  better  or 
worse,  according  to  your  own  criteria.  Are  there  any  questions? 
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C.1  INTERVIEW  QUESTIONNAIRE 


Subject:  _ 

Run:  _ 

1.  My  initial  return  to  and  steady -up  on  the  channel  centerline  was:  (check  one) 

_  poor 

_  fair 

_  good 

_  excellent 

2.  During  this  initial  maneuver,  I:  (check  any) 

_  overshot  the  centerline 

_  never  reached  the  centerline 

_  maneuvered  too  severely 

_  maneuvered  too  gradually 

3.  After  I  steadied  up,  my  transit  of  the  first  straight  leg  was:  (check  one) 

_  poor 

_  fair 

_  good 

_  excellent 

4.  Except  for  my  initial  offset,  in  the  first  straight  leg  I  was:  (check  one) 

_  never  on  the  centerline 

_  seldom  on  the  centerline 

_  usually  on  the  centerline 

_  always  on  the  centerline 

5.  In  the  first  straight  leg,  I  had  difficulty:  (check  any) 

_  determining  a  proper  course 

_  maintaining  a  steady  course 

_  keeping  on  the  centerline 

_  judging  my  speed 

6.  My  overall  turn  at  the  bend  was:  (check  one) 

_  poor 

_  fair 

_  good 

_  excellent 

7.  My  turn  rudder  before  the  bend  should  have  been:  (check  any) 

-  earlier  - 

_  later 
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larger 

smaller 


S.  I  entered  the  bend:  (check  any) 

_  too  fast 

_  too  slow 

_  too  far  to  the  right  side  of  the  channel 

_  too  far  to  the  left  side  of  the  channel 

9.  My  turn  through  the  bend  resulted  in:  (checlcany) 

_  an  overshoot  of  the  bend 

_  an  undershoot  of  the  bend 

_  excessive  rate  of  turn  into  the  bend 

_  excessive  drift  angle  (crabbing)  through  the  bend 

10.  Following  the  bend,  my  steady-up  on  the  second  leg  was 

_  poor 

_  fair 

_  good 

_  excellent 

11.  My  check  rudder  after  the  bend  should  have  been:  (check  any) 

_  earlier 

_  later 

_  larger 

_  smaller 

12.  In  steadying  up  beyond  the  bend,  I:  (check  any) 

_  overshot  the  centerline 

_  never  reached  the  centerline 

_  may  have  gone  outside  the  channel 

_  definitely  went  outside  the  channel 

1 3.  After  I  steadied  up,  my  transit  of  the  second  straight  leg  was:  (check  one) 

_  poor 

_  fair 

_  good 

_  excellent 

14.  In  the  second  straight  leg,  I  was:  (check  one) 

_  never  on  the  centerline 

_  seldom  on  the  centerline 

_  usually  on  the  centerline 

_  always  on  the  centerline 

15.  In  the  second  straight  leg,  I  had  difficulty:  (check  any) 

_  determining  a  proper  course 
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_  maintaining  a  steady  course 

_  keeping  on  the  centerline 

_  judging  my  speed 

16.  In  this  run,  the  displayed  information  was:  (check  one) 

_  never  accurate 

_  sometimes  accurate 

_  usually  accurate 

_  always  accurate 

17.  The  rate  of  speed  at  which  the  information  on  the  display  was  updated  was: 
(check  one) 

_  too  slow 

_  slow  but  acceptable 

_  fast  but  acceptable 

_  too  fast 

18.  In  this  run,  the  navigation  display:  (check  any) 

_  flickered 

_  jittered 

_  drifted 

_  was  cluttered 

_  was  erratic  or  unreliable 

_  should  have  been  on  a  different  range  scale 

_  was  difficult  to  understand 

19.  In  general,  the  overall  pilotage  was:  (check  one) 

_  poor 

_  fair 

_  good 

excellent 

20.  Given  another  opportunity  with  this  same  display,  ship,  and  waterway,  I  could  do 
an  even  better  job  of  staying  in  the  channel  centerline:  (check  one) 

_  yes 

_  no 
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C.2  QUESTIONNAIRE  RESULTS  BY  PERCENTAGE  OF  RESPONSES 


16  meter  rms  noise 
Tracker  Rise  Time  (Sec.) 


32  meter  rms  noise 
Tracker  Rise  Time  (Sec.) 
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My  initial  return  to  and  steady-up 
on  the  channel  centerline  was: 
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poor 

fair 
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During  this  initial  maneuver,  I: 
maneuvered  correctly 
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overshot  the  centerline 
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never  reached  the  centerline 
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maneuvered  too  gradually 

did  not  know 

After  I  steadied  up,  my  transit  of 
the  first  straight  leg  was: 


12.5 

12.5 

25.0 

[501 

25.0 

38.0 

50.0 

poor 

fair 

75.0 

[301 

38.0 

BS3 

50.0 

50.0 

good 

12.0 

12.0 

12.0 

12.0 

excellent 

Except  for  my  initial  offset,  in 
the  first  straight  leg  I  was: 


never  on  the  centerline 


12.0 

25.0 

12.0 

50.0 

25.0 

12.5 

seldom  on  the  centerline 

Ixft.oi 

[35151 

ms 

U£cj 

50.0 

75.0 

rcr 

• 

o 

o 

175751 

usually  on  the  centerline 

12.0 

12.0 

12.5 

always  on  the  centerline 

In  the  first  straight  leg,  I  had 

difficulty: 

SO] 

l^o.ol 

GoS' 

(ZXS  1 

1301  25.0 

H375! 

25.0 

none 

12.5 

4.0 

12.5 

12.0 

determining  a  proper  course 

31.0 

38.0 

25.0 

24.0 

12.5 

25.0 

[45751 

maintaining  a  steady  course 

12.0 

25.0 

25.0 

34.0 

150.51 

31.0 

19.0 

keeping  on  the  centerline 

6.5 

6.0 

judging  my  speed 

My  overall  turn  at  the  bend  was: 

25.0 

[301 

38.0 

12.0 

ES3 

flO 

(63751 

25.0 

poor 

25.0 

l507ol 

25.0 

25.0 

12.5 

150 

fair 

f50751 

[SOj 

38.0 

25.0 

[37751 

12.5 

25.0 

good 

12.0 

12.0 

12.5 

12.0 

excellent 
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V. 


C‘.2  (con't) 


16  meter  rms  noise 

32  meter  rms  noise 

Tracker  Rise 

Time  (Sec.) 

Tracker  Rise  Time  (Sec.) 

3 

12 

24 

24* 

3 

12 

24 

24* 

My  turn  rudder  before  the  bend 
should  have  been: 

12.5 

si 

25.0 

HPl 

25.0 

[501 

12.5 

12.0 

none  (was  appropriate) 

136.61 

31.0 

31.0 

25.0 

144351 

2^0 

earlier 

12.5 

12.5 

12.5 

25.0 

6.0 

31.5 

later 

25.0 

144351 

25.0 

12.0 

Up] 

31.0 

31.0 

31.5 

larger 

12.5 

6.0 

12.5 

smaller 

l63.nl 

I  entered  the  bend: 

150-01 

fftTnl 

£12 

175.01  HO  1 38.01 

appropriately 

12.0 

12.5 

12.5 

12.0 

12.5 

12.0 

12.0 

too  fast 

44.0 

12.5 

12.5 

25.0 

31.0 

too  slow 

25.0 

12.5 

12.0 

too  far  to  the  right  side  of  the 

channel 

6.0 

12.5 

25.0 

25.0 

19.0 

too  far  to  the  left  side  of  the 
channel 

My  turn  through  the  bend  resulted 
in: 

|  none  (was  appropriate) 

12.5 

38.0 

25.0 

12.0 

25.0 

12.0 

5£o 

156.01 

mM 

[44^ 

[55351 

□p 

C&3 

reoi 

31.0 

an  overshoot  of  the  bend 

38.0 

12.0 

12.5 

12.0 

12.5 

12.5 

an  undershoot  of  the  bend 
excessive  rate  of  turn  into  the 

bend 

31.5 

12.0 

6.0 

6.5 

eO 

12.5 

19.0 

25.0 

excessive  drift  angle  (crabb¬ 

ing)  through  the  bend) 

Following  the  bend,  my  steady-up 
on  the  second  leg  was 


12.0 

25.0 

12.0 

25.0 

25.0 

25.0 

25.0 

poor 

25.0 

37.0 

12.0 

38.0 

f5o3o! 

12.0 

12.5 

fair 

rS330l 

163.0! 

f63351 

38.0' 

25.0 

[55351 

m\ 

[75351 

good 

12.0 

12.5 

excellent 

My  check  rudder  after  the  bend 

should  have  been: 

f353ol 

cm 

IIO  I503q1  12.0 

138351 

25.0 

12.0 

none  (was  appropriate) 

6.5 

25.0 

31.0 

25.0 

138.01 

31.0 

12,0 

earlier 

25.0 

18.5 

12.5 

19.0 

12.5 

EO] 

[55351 

later 

6.5 

18.5 

31.0 

25.0 

6.0 

12.0 

19.0 

larger 

12.0 

12.5 

6.0 

12.5 

25.0 

smaller 
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C-S 


V, 


C.2  (con't) 


16  meter  rms  noise 
Tracker  Rise  Time  (Sec.) 

3  12  24  24* 


38.0 

38.0 

135751 

38.0 

144751 

38.0 

25.0 

38.0 

1575 

12.5 

1575 

6.0 

12.0 

12.0 

12.5 

12.0 

32  meter  rms  noise 


Tracker  Rise 

Time 

(sec.) 

3 

12 

24 

24* 

12.5 

25.0 

eh 

25.0 

B5751 

31.0 

[38751 

25.0 

25.0 

12.5 

19.0 

12.0 

12.5 

12.0 

- 1 

12.0 

12.0 

25.0 

37.5i 

38.0 

25.0 

38.0  150.01 

25.0 

38.0 

25.0 

37.51 

[3575 

[63751 

[55751 

38.0 

I63.0i 

135751 

HO 

25.0 

12.0 

12.0 

12.0 

12.0 

12.0 

12.0 

12.0 

25.0 

50.0 

25.0 

50.0 

25.0 

188751 

100.0 

[UToj 

H3751 

12.0 

50.0 

75.0 

50.0 

tiMl 

12.0 

25.0  HOI 
12.5 

lUToi  31.0 
25.0  31.0 

6.5 


38.0 

EH 

12.0 

12.0 

[4475 

6.0 

38.0 

HO 

6.0  19.0 

30.0  31.0 

160751  6.0 

4.0  6.0 


25.0 

25.0 

6.0  44.0 

[401  1301 


25.0 


25.0 


_ 12.0 

37.5 

1 100.0!  [53751  [73751 

1 - 1  25X)  25.0 

37.5 

100.0 

25.0 


12.5 

[4575] 

no 

12.0 

12.0 

25.0 

12.5 

12.5 

12.5 

12.0 

38.0 

25.0 

25.0 

37.5 

[73751 

25.0 

no 

[35751 

163.01 

[5375 

37.5 

12.5 

12.5 

12.0 

12.0 

In  steadying  up  beyond  the  bend,  I: 
none  (was  on  the  centerline) 
overshot  the  centerline 
never  reached  the  centerline 
may  have  gone  outside  the 
channel 

definitely  went  outside  the 
channel 

After  1  steadied  up,  my  transit  of 
the  second  straight  leg  was: 
poor 
fair 
good 

excellent 

In  the  second  straight  leg,  I  was: 
never  on  the  centerline 
seldom  on  the  centerline 
usually  on  the  centerline 
always  on  the  centerline 

In  the  second  straight  leg,  I  had 
difficulty: 
none 

determining  a  proper  course 
maintaining  a  steady  course 
keeping  on  the  centerline 
judging  my  speed 

In  this  run,  the  displayed  informa¬ 
tion  was: 

never  accurate 
sometimes  accurate 
usually  accurate 
always  accurate 

The  rate  of  speed  at  which  the 
information  on  the  display  was 
updated  was: 
acceptable 
too  slow 

slow  but  acceptable 
fast  but  acceptable 
too  fast 
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C.2  (con't) 

16  meter  rms  32  meter  rms  noise 

Tracker  Rise  Time  (Sec.)  Tracker  Rise  Time  (sec.) 

3  12  24  24*  3  12  24  24* 


6.0 

12.5  12.0 

17.0  19.0  25.0  6.0 

4.0  6.0  _ 

21.0  19.0  12.5  1401 


12.0  [301  3S.0 


12.5 

12.5 

EH 

6.0 

roi 

6.0 

11.0 

19.5 

6.0 

17.0 

25.0 

23.0 

\m 

23.0 

6.0 

19.5 

5.0 

|?[.o|  [29751 

12.0 

5.0 

12.5 

12.0 

In  this  run,  the  navigation  display: 
erratic 
flickered 
jittered 
drifted 
was  cluttered 
was  erratic  or  unreliable 
should  have  been  on  a  differ¬ 
ent  range  scale 
was  difficult  to  understand 
acceptable 


12.0  12.0  12.0  12.5  |  37.5  3S.0  _ 

25.0  38.0  38.0  12.5  |  37.5  [62151 

f6X0l  HOI  [501  [7P1  25.0  [62751  38.0 


In  general,  the  overall  pilotage 
was: 
poor 
fair 
good 

excellent 


Given  another  opportunity  with 
this  same  display,  ship,  and  water¬ 
way,  I  could  do  an  even  better  job 
of  staying  in  the  channel  center- 
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APPENDIX  D 


MANEUVERING  ANALYSIS  RATIONALE 


Appendix  D  contains  the  rationaie  and  methodology  for  determining  the  measures 
shown  on  the  "Effect  on  Maneuvering  Performance"  tables  presented  in  Section  5. 

MANEUVERING  ANALYSIS 
CRITERIA  FOR  DETERMINING: 

1.  Distance  to  return  to  centerline 

a.  Alongtrack  distance  (ATD)  from  the  start  of  the  run  until  ownship  crosses  the 
centerline 

or  in  the  event  ownship  does  not  cross  centerline 

b.  ATD  from  the  start  of  the  run  until  ownship  makes  its  first  closest  approach 
to  the  centerline 

or  in  the  event  ownship  closest  approach  to  centerline  occurs  more  than 

1  nm  from  the  start  —  i-e.,  a  very  gradual  return  to  the  centerline 

c.  ATD  from  the  start  of  the  run  to  the  first  course  command  340  degrees  or 
larger 

or  in  the  event  no  course  command  340  degrees  or  larger  is  given 

d.  ATD  from  the  start  of  the  run  to  the  first  time  three  similar  crosstrack 
distances  occur  consecutively  (+  2  feet) 


2.  Overshoot  following  return  to  centerline 

a.  Largest  crosstrack  distance  (CTD)  immediately  following  when  ownship 
crosses  the  centerline 

or  in  the  event  ownship  does  not  cross  the  centerline 

b.  Largest  CTD  immediately  following  when  ownship  makes  its  first  closest 
approach  to  the  centerline 

or  in  the  event  a  340-degree  or  larger  course  command  or  three 
consecutive  similar  crosstrack  distances  are  used  to  establish  the  return 
to  centerline 

c.  CTD  at  the  time  of  "return  to  the  centerline" 


3.  Distance  to  steady  up  in  entrance  leg 

a.  ATD  from  the  start  of  the  run  when  crosstrack  distance  first  becomes  25  feet 
or  less  beyond  the  established  "overshoot" 
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or  in  the  event  crosstrack  distance  following  the  overshoot  does  not 
become  25  feet  or  less  within  1  nm  from  the  start,  t.e.,  a  very  gradual 
return  to  the  centerline 

b.  ATD  from  the  start  of  the  run  to  the  first  time  three  similar  crosstrack 
distances  occur  consecutively  (-2  feet) 

or  in  the  event  three  similar  crosstrack  distances  do  not  occur 
consecutively 

c.  ATD  from  the  start  of  the  run  but  beyond  1  nm  from  the  start  when 
crosstrack  distance  first  becomes  25  feet  or  less 


*».  Distance  to  steady  up  in  exit  leg 

a.  ATD  from  the  waypoint  (center  of  the  bend)  when  crosstrack  distance  is  less 
than  25  feet  and  proceeded  by  three  consecutive  course  errors  less  than  5  degrees, 
within  1  nm  from  the  waypoint 


or  in  the  event  crosstrack  distance  does  not  become  25  feet  or  less 
within  1  nm  from  the  waypoint 

b.  ATD  from  the  waypoint  (center  of  the  bend)  for  the  third  of  three 
consecutive  course  errors  less  than  5  degrees  after  the  closest  approach  to  the 
centerline,  excluding  the  initial  crossing  if  an  overshoot 

or  in  the  event  there  are  no  three  consecutive  course  errors  less  than  5 
degrees 

c.  The  third  of  the  three  smallest  consecutive  course  errors  beyond  1  nm  from 
the  waypoint 


5.  Distance  before  bend  at  inital  rudder 

a.  Distance  before  the  waypoint  (center  of  the  bend)  at  which  the  initial  turn 
rudder  was  applied. 


6.  Magnitude  of  initial  turn  rudder 

a.  The  amount  of  rudder  initially  applied  to  execute  the  turn. 

7.  Maximum  initial  turn  rudder 

a.  The  largest  rudder  applied  at  any  time  to  increase  the  swing 

S.  Frequency  of  turn  rudder  actuations 

a.  Total  number  of  rudder  actuations  prior  to  check  rudder 
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9.  Distance  beyond  bend  of  initial  check  rudder 

a.  Distance  beyond  the  waypoint  (center  of  the  bend)  at  which  the  initial  turn 
rudder  was  applied. 


10.  Magnitude  of  initial  check  rudder 

a.  The  amount  of  rudder  initially  applied  to  execute  the  turn. 
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APPENDIX  E 


STATISTICAL  COMPARISON  OF  TRACKKEEPING  PERFORMANCE 


Appendix  E  contains  the  statistical  comparison  of  mean  tracks  and  trackkeeping 
variability  at  each  data  line  of  the  waterway  between  display  design  variables.  This 
analysis  forms  the  basis  for  the  interpretations  of  pilotage  performance  of  Section  5 
and  is  frequently  referenced  in  that  section. 

The  figures  presented  herein  were  derived  as  a  result  of  statistical  tests  on  the 
means  and  standard  deviations  of  crosstrack  distances  for  all  runs  conducted  within 
each  experimental  condition.  The  T-test  was  used  to  test  for  differences  in  group 
means  and  the  F-statistic  for  differences  in  group  variability.  Differences  greater 
than  or  equal  to  the  0.10  level  of  significance  are  indicated  by  arrows  along  the 
trackiine.  The  data  are  used  as  statistical  support  in  Section  5.  Data  lines  are 
numbered  in  both  directions  from  the  center  of  the  bend  along  the  entrance  leg  1 
and  exit  leg  2.  They  represent  discrete  measurement  points  474  feet  apart  to  which 
each  statistical  test  was  applied. 
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Figure  E-l.  Statistical  Comparison  on  the  Effects  of  Noise  with  ail  Trackers 


Figure  E-2.  Statistical  Comparison  between  Trackers  with  3-Second  and  12-Second 

Rise  Time  in  16  Meter  RMS  Noise 


Statistical  Comparison  between  Trackers  with  3-Second  and  12-Second 
Rise  Time  in  32  Meter  RMS  Noise 


Figure  E-4.  Statistical  Comparison  between  Trackers  with  12-Second  and  24-Second 

Rise  Time  in  16  Meter  RMS  Noise 


Statistical  Comparison  between  Trackers  with  12-Second  and  24-Second 
Rise  Time  in  32  Meter  RMS  Noise 


Statistical  Comparison  between  Trackers  with  3-Second  and  24-Second 
Rise  Time  in  16  Meter  RMS  Noise 


Statistical  Comparison  between  Trackers  with  3-Second  and  24-Second 
Rise  Time  in  32  Meter  RMS  Noise 
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Figure  E-9.  Statistical  Comparison  between  Unaided  and  Gyro-Aided  Trackers  in  32  Meter  RMS  Noise 


